From nobody Sat Oct 4 22:35:30 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 154122D2385 for ; Wed, 13 Aug 2025 05:49:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755064169; cv=none; b=MLxDFvy+WypSeqmxsbmo067yuj+C975uN5SPGqL587zwb+EFC44VjLuEh7KCPOlrJH5v7dQKDQLAAZp9fHH/BobCQ6qCmPc0iIZdUmLiuEjr5lK11hX803hmNM6orTy6H04EvXvdxbKTPnip2cXj1g1d6qfVwhcxHbZ7QCy7YZA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755064169; c=relaxed/simple; bh=ct2P4Jylji6l95HatANo7+Nm1zSSO67jYR8EPb0kjMg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=W+Tv59XE8lRMcH/WZL4UJ2U03NCuj8auyvGQjliSVPNUabTVDOFLoDfXi3h7Yq8rRbnnxxq/VTSVyUmDfoNmc9s+mOy8yjckaRbQQTpj959t2MRgaavHS71tuLDjYe4S4aFe4becSXpxbOpKmqs5Ke6gYBuehtHQaJB6fvfyJV4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Gs04jd+o; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Gs04jd+o" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1755064166; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MrnLAqNPP3MNN09OO/tPRCXaRUs0p44l+QLGU8ZWbJk=; b=Gs04jd+oBnfdtoEit/0FTB3U8eTEq9odnQJHiXGoyvMBHedFiAjKvirpPixZ9C67k7zSk5 Sacax/9bND2YH/wt7wqjULhA+iMAkkf6xgpRHAeLvVDw0/zrjJv0pv6uVyNCN1/89tsmXM qWw3X3IEhmGjkDpBuUm9o+r39GpNp2Y= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-249-kSQY00JZNVishG9zl_7usg-1; Wed, 13 Aug 2025 01:49:25 -0400 X-MC-Unique: kSQY00JZNVishG9zl_7usg-1 X-Mimecast-MFC-AGG-ID: kSQY00JZNVishG9zl_7usg_1755064164 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4676A1956095; Wed, 13 Aug 2025 05:49:24 +0000 (UTC) Received: from localhost.localdomain (unknown [10.72.112.168]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 444581800280; Wed, 13 Aug 2025 05:49:19 +0000 (UTC) From: Jason Wang To: mst@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, eperezma@redhat.com, virtualization@lists.linux.dev, linux-kernel@vger.kernel.org Cc: hch@infradead.org Subject: [PATCH V5 9/9] vduse: switch to use virtio map API instead of DMA API Date: Wed, 13 Aug 2025 13:48:31 +0800 Message-ID: <20250813054831.25865-10-jasowang@redhat.com> In-Reply-To: <20250813054831.25865-1-jasowang@redhat.com> References: <20250813054831.25865-1-jasowang@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" Lacking the support of device specific mapping supported in virtio, VDUSE must trick the DMA API in order to make virtio-vdpa transport work. This is done by advertising vDPA device as dma device with a VDUSE specific dma_ops even if it doesn't do DMA at all. This will be fixed by this patch. Thanks to the new mapping operations support by virtio and vDPA. VDUSE can simply switch to advertise its specific mappings operations to virtio via virtio-vdpa then DMA API is not needed for VDUSE any more and iova domain could be used as the mapping token instead. Signed-off-by: Jason Wang --- drivers/vdpa/Kconfig | 8 +-- drivers/vdpa/vdpa_user/iova_domain.c | 2 +- drivers/vdpa/vdpa_user/iova_domain.h | 2 +- drivers/vdpa/vdpa_user/vduse_dev.c | 75 ++++++++++++++-------------- 4 files changed, 41 insertions(+), 46 deletions(-) diff --git a/drivers/vdpa/Kconfig b/drivers/vdpa/Kconfig index 559fb9d3271f..857cf288c876 100644 --- a/drivers/vdpa/Kconfig +++ b/drivers/vdpa/Kconfig @@ -34,13 +34,7 @@ config VDPA_SIM_BLOCK =20 config VDPA_USER tristate "VDUSE (vDPA Device in Userspace) support" - depends on EVENTFD && MMU && HAS_DMA - # - # This driver incorrectly tries to override the dma_ops. It should - # never have done that, but for now keep it working on architectures - # that use dma ops - # - depends on ARCH_HAS_DMA_OPS + depends on EVENTFD && MMU select VHOST_IOTLB select IOMMU_IOVA help diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/= iova_domain.c index 58116f89d8da..ccaed24b7ef8 100644 --- a/drivers/vdpa/vdpa_user/iova_domain.c +++ b/drivers/vdpa/vdpa_user/iova_domain.c @@ -447,7 +447,7 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *= domain, =20 void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, size_t size, dma_addr_t *dma_addr, - gfp_t flag, unsigned long attrs) + gfp_t flag) { struct iova_domain *iovad =3D &domain->consistent_iovad; unsigned long limit =3D domain->iova_limit; diff --git a/drivers/vdpa/vdpa_user/iova_domain.h b/drivers/vdpa/vdpa_user/= iova_domain.h index 7f3f0928ec78..1f3c30be272a 100644 --- a/drivers/vdpa/vdpa_user/iova_domain.h +++ b/drivers/vdpa/vdpa_user/iova_domain.h @@ -64,7 +64,7 @@ void vduse_domain_unmap_page(struct vduse_iova_domain *do= main, =20 void *vduse_domain_alloc_coherent(struct vduse_iova_domain *domain, size_t size, dma_addr_t *dma_addr, - gfp_t flag, unsigned long attrs); + gfp_t flag); =20 void vduse_domain_free_coherent(struct vduse_iova_domain *domain, size_t s= ize, void *vaddr, dma_addr_t dma_addr, diff --git a/drivers/vdpa/vdpa_user/vduse_dev.c b/drivers/vdpa/vdpa_user/vd= use_dev.c index f161059d543e..3260edefdf0d 100644 --- a/drivers/vdpa/vdpa_user/vduse_dev.c +++ b/drivers/vdpa/vdpa_user/vduse_dev.c @@ -88,6 +88,7 @@ struct vduse_dev { struct device *dev; struct vduse_virtqueue **vqs; struct vduse_iova_domain *domain; + struct vduse_iova_domain *dom; char *name; struct mutex lock; spinlock_t msg_lock; @@ -814,59 +815,53 @@ static const struct vdpa_config_ops vduse_vdpa_config= _ops =3D { .free =3D vduse_vdpa_free, }; =20 -static void vduse_dev_sync_single_for_device(struct device *dev, +static void vduse_dev_sync_single_for_device(void *token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_dev *vdev =3D dev_to_vduse(dev); - struct vduse_iova_domain *domain =3D vdev->domain; + struct vduse_iova_domain *domain =3D token; =20 vduse_domain_sync_single_for_device(domain, dma_addr, size, dir); } =20 -static void vduse_dev_sync_single_for_cpu(struct device *dev, +static void vduse_dev_sync_single_for_cpu(void *token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir) { - struct vduse_dev *vdev =3D dev_to_vduse(dev); - struct vduse_iova_domain *domain =3D vdev->domain; + struct vduse_iova_domain *domain =3D token; =20 vduse_domain_sync_single_for_cpu(domain, dma_addr, size, dir); } =20 -static dma_addr_t vduse_dev_map_page(struct device *dev, struct page *page, +static dma_addr_t vduse_dev_map_page(void *token, struct page *page, unsigned long offset, size_t size, enum dma_data_direction dir, unsigned long attrs) { - struct vduse_dev *vdev =3D dev_to_vduse(dev); - struct vduse_iova_domain *domain =3D vdev->domain; + struct vduse_iova_domain *domain =3D token; =20 return vduse_domain_map_page(domain, page, offset, size, dir, attrs); } =20 -static void vduse_dev_unmap_page(struct device *dev, dma_addr_t dma_addr, +static void vduse_dev_unmap_page(void *token, dma_addr_t dma_addr, size_t size, enum dma_data_direction dir, unsigned long attrs) { - struct vduse_dev *vdev =3D dev_to_vduse(dev); - struct vduse_iova_domain *domain =3D vdev->domain; + struct vduse_iova_domain *domain =3D token; =20 return vduse_domain_unmap_page(domain, dma_addr, size, dir, attrs); } =20 -static void *vduse_dev_alloc_coherent(struct device *dev, size_t size, - dma_addr_t *dma_addr, gfp_t flag, - unsigned long attrs) +static void *vduse_dev_alloc_coherent(void *token, size_t size, + dma_addr_t *dma_addr, gfp_t flag) { - struct vduse_dev *vdev =3D dev_to_vduse(dev); - struct vduse_iova_domain *domain =3D vdev->domain; + struct vduse_iova_domain *domain =3D token; unsigned long iova; void *addr; =20 *dma_addr =3D DMA_MAPPING_ERROR; addr =3D vduse_domain_alloc_coherent(domain, size, - (dma_addr_t *)&iova, flag, attrs); + (dma_addr_t *)&iova, flag); if (!addr) return NULL; =20 @@ -875,31 +870,45 @@ static void *vduse_dev_alloc_coherent(struct device *= dev, size_t size, return addr; } =20 -static void vduse_dev_free_coherent(struct device *dev, size_t size, - void *vaddr, dma_addr_t dma_addr, - unsigned long attrs) +static void vduse_dev_free_coherent(void *token, size_t size, + void *vaddr, dma_addr_t dma_addr, + unsigned long attrs) { - struct vduse_dev *vdev =3D dev_to_vduse(dev); - struct vduse_iova_domain *domain =3D vdev->domain; + struct vduse_iova_domain *domain =3D token; =20 vduse_domain_free_coherent(domain, size, vaddr, dma_addr, attrs); } =20 -static size_t vduse_dev_max_mapping_size(struct device *dev) +static bool vduse_dev_need_sync(void *token, dma_addr_t dma_addr) { - struct vduse_dev *vdev =3D dev_to_vduse(dev); - struct vduse_iova_domain *domain =3D vdev->domain; + struct vduse_iova_domain *domain =3D token; + + return dma_addr < domain->bounce_size; +} + +static int vduse_dev_mapping_error(void *token, dma_addr_t dma_addr) +{ + if (unlikely(dma_addr =3D=3D DMA_MAPPING_ERROR)) + return -ENOMEM; + return 0; +} + +static size_t vduse_dev_max_mapping_size(void *token) +{ + struct vduse_iova_domain *domain =3D token; =20 return domain->bounce_size; } =20 -static const struct dma_map_ops vduse_dev_dma_ops =3D { +static const struct virtio_map_ops vduse_map_ops =3D { .sync_single_for_device =3D vduse_dev_sync_single_for_device, .sync_single_for_cpu =3D vduse_dev_sync_single_for_cpu, .map_page =3D vduse_dev_map_page, .unmap_page =3D vduse_dev_unmap_page, .alloc =3D vduse_dev_alloc_coherent, .free =3D vduse_dev_free_coherent, + .need_sync =3D vduse_dev_need_sync, + .mapping_error =3D vduse_dev_mapping_error, .max_mapping_size =3D vduse_dev_max_mapping_size, }; =20 @@ -2003,27 +2012,18 @@ static struct vduse_mgmt_dev *vduse_mgmt; static int vduse_dev_init_vdpa(struct vduse_dev *dev, const char *name) { struct vduse_vdpa *vdev; - int ret; =20 if (dev->vdev) return -EEXIST; =20 vdev =3D vdpa_alloc_device(struct vduse_vdpa, vdpa, dev->dev, - &vduse_vdpa_config_ops, NULL, + &vduse_vdpa_config_ops, &vduse_map_ops, 1, 1, name, true); if (IS_ERR(vdev)) return PTR_ERR(vdev); =20 dev->vdev =3D vdev; vdev->dev =3D dev; - vdev->vdpa.dev.dma_mask =3D &vdev->vdpa.dev.coherent_dma_mask; - ret =3D dma_set_mask_and_coherent(&vdev->vdpa.dev, DMA_BIT_MASK(64)); - if (ret) { - put_device(&vdev->vdpa.dev); - return ret; - } - set_dma_ops(&vdev->vdpa.dev, &vduse_dev_dma_ops); - vdev->vdpa.mapping_token.dma_dev =3D &vdev->vdpa.dev; vdev->vdpa.mdev =3D &vduse_mgmt->mgmt_dev; =20 return 0; @@ -2056,6 +2056,7 @@ static int vdpa_dev_add(struct vdpa_mgmt_dev *mdev, c= onst char *name, return -ENOMEM; } =20 + dev->vdev->vdpa.mapping_token.token =3D dev->domain; ret =3D _vdpa_register_device(&dev->vdev->vdpa, dev->vq_num); if (ret) { put_device(&dev->vdev->vdpa.dev); --=20 2.31.1