From nobody Mon May  5 15:54:24 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
	dkim=fail;
	spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
 permitted sender)
  smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
	dmarc=fail(p=none dis=none)  header.from=redhat.com
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
 mx.zohomail.com
	with SMTPS id 1647325516318386.7432618499455;
 Mon, 14 Mar 2022 23:25:16 -0700 (PDT)
Received: from localhost ([::1]:40768 helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <qemu-devel-bounces+importer=patchew.org@nongnu.org>)
	id 1nU0cL-0005qv-Eq
	for importer@patchew.org; Tue, 15 Mar 2022 02:25:13 -0400
Received: from eggs.gnu.org ([209.51.188.92]:40830)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <jasowang@redhat.com>)
 id 1nU0Sh-0001Om-Tw
 for qemu-devel@nongnu.org; Tue, 15 Mar 2022 02:15:23 -0400
Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:46732)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <jasowang@redhat.com>)
 id 1nU0Sf-0004bz-3J
 for qemu-devel@nongnu.org; Tue, 15 Mar 2022 02:15:15 -0400
Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com
 [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS
 (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
 us-mta-664-5B5o6x29M-a_MGR-3iS-og-1; Tue, 15 Mar 2022 02:15:09 -0400
Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com
 [10.11.54.2])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BCF4F101AA56;
 Tue, 15 Mar 2022 06:15:08 +0000 (UTC)
Received: from localhost.localdomain (ovpn-14-38.pek2.redhat.com
 [10.72.14.38])
 by smtp.corp.redhat.com (Postfix) with ESMTP id 4E82640C128B;
 Tue, 15 Mar 2022 06:15:06 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
 s=mimecast20190719; t=1647324912;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=6Sw80/7CcCf1QrcFkqWFtGChjRfHbdAq/r4T+u9LZ0k=;
 b=fA2TOPf1LB1wmBTDzVtVXQ4+D5Sy87KD2DB237S/QlEPrD7aexo+VMp0V294aj3lxRjDMu
 2ntHzXZZC4Xb+pgStxLzpQQwOgDoW2F08MWuaXLOA7UbmFvAQD1XLZ5u1ToSKA029Dz57q
 zOVo63dJf3QT4+oOT7JbzilTxDcjOZE=
X-MC-Unique: 5B5o6x29M-a_MGR-3iS-og-1
From: Jason Wang <jasowang@redhat.com>
To: qemu-devel@nongnu.org,
	peter.maydell@linaro.org
Subject: [PULL V3 12/15] vdpa: Add custom IOTLB translations to SVQ
Date: Tue, 15 Mar 2022 14:14:17 +0800
Message-Id: <20220315061420.31910-13-jasowang@redhat.com>
In-Reply-To: <20220315061420.31910-1-jasowang@redhat.com>
References: <20220315061420.31910-1-jasowang@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2
Authentication-Results: relay.mimecast.com;
 auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
 as permitted sender) client-ip=209.51.188.17;
 envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
 helo=lists.gnu.org;
Received-SPF: pass client-ip=170.10.133.124;
 envelope-from=jasowang@redhat.com;
 helo=us-smtp-delivery-124.mimecast.com
X-Spam_score_int: -21
X-Spam_score: -2.2
X-Spam_bar: --
X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082,
 DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001,
 T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
 <mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Cc: =?UTF-8?q?Eugenio=20P=C3=A9rez?= <eperezma@redhat.com>,
 Jason Wang <jasowang@redhat.com>, "Michael S . Tsirkin" <mst@redhat.com>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: "Qemu-devel" <qemu-devel-bounces+importer=patchew.org@nongnu.org>
X-ZohoMail-DKIM: fail (Header signature does not verify)
X-ZM-MESSAGEID: 1647325517689100001

From: Eugenio P=C3=A9rez <eperezma@redhat.com>

Use translations added in VhostIOVATree in SVQ.

Only introduce usage here, not allocation and deallocation. As with
previous patches, we use the dead code paths of shadow_vqs_enabled to
avoid commiting too many changes at once. These are impossible to take
at the moment.

Signed-off-by: Eugenio P=C3=A9rez <eperezma@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 hw/virtio/vhost-shadow-virtqueue.c |  86 +++++++++++++++++++++++---
 hw/virtio/vhost-shadow-virtqueue.h |   6 +-
 hw/virtio/vhost-vdpa.c             | 122 +++++++++++++++++++++++++++++++--=
----
 include/hw/virtio/vhost-vdpa.h     |   3 +
 4 files changed, 187 insertions(+), 30 deletions(-)

diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi=
rtqueue.c
index ece50b8..b232803 100644
--- a/hw/virtio/vhost-shadow-virtqueue.c
+++ b/hw/virtio/vhost-shadow-virtqueue.c
@@ -70,7 +70,59 @@ static uint16_t vhost_svq_available_slots(const VhostSha=
dowVirtqueue *svq)
     return svq->vring.num - (svq->shadow_avail_idx - svq->shadow_used_idx);
 }
=20
-static void vhost_vring_write_descs(VhostShadowVirtqueue *svq,
+/**
+ * Translate addresses between the qemu's virtual address and the SVQ IOVA
+ *
+ * @svq: Shadow VirtQueue
+ * @vaddr: Translated IOVA addresses
+ * @iovec: Source qemu's VA addresses
+ * @num: Length of iovec and minimum length of vaddr
+ */
+static bool vhost_svq_translate_addr(const VhostShadowVirtqueue *svq,
+                                     hwaddr *addrs, const struct iovec *io=
vec,
+                                     size_t num)
+{
+    if (num =3D=3D 0) {
+        return true;
+    }
+
+    for (size_t i =3D 0; i < num; ++i) {
+        DMAMap needle =3D {
+            .translated_addr =3D (hwaddr)(uintptr_t)iovec[i].iov_base,
+            .size =3D iovec[i].iov_len,
+        };
+        Int128 needle_last, map_last;
+        size_t off;
+
+        const DMAMap *map =3D vhost_iova_tree_find_iova(svq->iova_tree, &n=
eedle);
+        /*
+         * Map cannot be NULL since iova map contains all guest space and
+         * qemu already has a physical address mapped
+         */
+        if (unlikely(!map)) {
+            qemu_log_mask(LOG_GUEST_ERROR,
+                          "Invalid address 0x%"HWADDR_PRIx" given by guest=
",
+                          needle.translated_addr);
+            return false;
+        }
+
+        off =3D needle.translated_addr - map->translated_addr;
+        addrs[i] =3D map->iova + off;
+
+        needle_last =3D int128_add(int128_make64(needle.translated_addr),
+                                 int128_make64(iovec[i].iov_len));
+        map_last =3D int128_make64(map->translated_addr + map->size);
+        if (unlikely(int128_gt(needle_last, map_last))) {
+            qemu_log_mask(LOG_GUEST_ERROR,
+                          "Guest buffer expands over iova range");
+            return false;
+        }
+    }
+
+    return true;
+}
+
+static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, hwaddr *sg,
                                     const struct iovec *iovec, size_t num,
                                     bool more_descs, bool write)
 {
@@ -89,7 +141,7 @@ static void vhost_vring_write_descs(VhostShadowVirtqueue=
 *svq,
         } else {
             descs[i].flags =3D flags;
         }
-        descs[i].addr =3D cpu_to_le64((hwaddr)(intptr_t)iovec[n].iov_base);
+        descs[i].addr =3D cpu_to_le64(sg[n]);
         descs[i].len =3D cpu_to_le32(iovec[n].iov_len);
=20
         last =3D i;
@@ -104,6 +156,8 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue *s=
vq,
 {
     unsigned avail_idx;
     vring_avail_t *avail =3D svq->vring.avail;
+    bool ok;
+    g_autofree hwaddr *sgs =3D g_new(hwaddr, MAX(elem->out_num, elem->in_n=
um));
=20
     *head =3D svq->free_head;
=20
@@ -114,9 +168,20 @@ static bool vhost_svq_add_split(VhostShadowVirtqueue *=
svq,
         return false;
     }
=20
-    vhost_vring_write_descs(svq, elem->out_sg, elem->out_num, elem->in_num=
 > 0,
-                            false);
-    vhost_vring_write_descs(svq, elem->in_sg, elem->in_num, false, true);
+    ok =3D vhost_svq_translate_addr(svq, sgs, elem->out_sg, elem->out_num);
+    if (unlikely(!ok)) {
+        return false;
+    }
+    vhost_vring_write_descs(svq, sgs, elem->out_sg, elem->out_num,
+                            elem->in_num > 0, false);
+
+
+    ok =3D vhost_svq_translate_addr(svq, sgs, elem->in_sg, elem->in_num);
+    if (unlikely(!ok)) {
+        return false;
+    }
+
+    vhost_vring_write_descs(svq, sgs, elem->in_sg, elem->in_num, false, tr=
ue);
=20
     /*
      * Put the entry in the available array (but don't update avail->idx u=
ntil
@@ -395,9 +460,9 @@ void vhost_svq_set_svq_call_fd(VhostShadowVirtqueue *sv=
q, int call_fd)
 void vhost_svq_get_vring_addr(const VhostShadowVirtqueue *svq,
                               struct vhost_vring_addr *addr)
 {
-    addr->desc_user_addr =3D (uint64_t)(intptr_t)svq->vring.desc;
-    addr->avail_user_addr =3D (uint64_t)(intptr_t)svq->vring.avail;
-    addr->used_user_addr =3D (uint64_t)(intptr_t)svq->vring.used;
+    addr->desc_user_addr =3D (uint64_t)(uintptr_t)svq->vring.desc;
+    addr->avail_user_addr =3D (uint64_t)(uintptr_t)svq->vring.avail;
+    addr->used_user_addr =3D (uint64_t)(uintptr_t)svq->vring.used;
 }
=20
 size_t vhost_svq_driver_area_size(const VhostShadowVirtqueue *svq)
@@ -518,11 +583,13 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq)
  * Creates vhost shadow virtqueue, and instructs the vhost device to use t=
he
  * shadow methods and file descriptors.
  *
+ * @iova_tree: Tree to perform descriptors translations
+ *
  * Returns the new virtqueue or NULL.
  *
  * In case of error, reason is reported through error_report.
  */
-VhostShadowVirtqueue *vhost_svq_new(void)
+VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree)
 {
     g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, =
1);
     int r;
@@ -543,6 +610,7 @@ VhostShadowVirtqueue *vhost_svq_new(void)
=20
     event_notifier_init_fd(&svq->svq_kick, VHOST_FILE_UNBIND);
     event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call);
+    svq->iova_tree =3D iova_tree;
     return g_steal_pointer(&svq);
=20
 err_init_hdev_call:
diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi=
rtqueue.h
index 38b3b91..e5e24c5 100644
--- a/hw/virtio/vhost-shadow-virtqueue.h
+++ b/hw/virtio/vhost-shadow-virtqueue.h
@@ -13,6 +13,7 @@
 #include "qemu/event_notifier.h"
 #include "hw/virtio/virtio.h"
 #include "standard-headers/linux/vhost_types.h"
+#include "hw/virtio/vhost-iova-tree.h"
=20
 /* Shadow virtqueue to relay notifications */
 typedef struct VhostShadowVirtqueue {
@@ -43,6 +44,9 @@ typedef struct VhostShadowVirtqueue {
     /* Virtio device */
     VirtIODevice *vdev;
=20
+    /* IOVA mapping */
+    VhostIOVATree *iova_tree;
+
     /* Map for use the guest's descriptors */
     VirtQueueElement **ring_id_maps;
=20
@@ -75,7 +79,7 @@ void vhost_svq_start(VhostShadowVirtqueue *svq, VirtIODev=
ice *vdev,
                      VirtQueue *vq);
 void vhost_svq_stop(VhostShadowVirtqueue *svq);
=20
-VhostShadowVirtqueue *vhost_svq_new(void);
+VhostShadowVirtqueue *vhost_svq_new(VhostIOVATree *iova_tree);
=20
 void vhost_svq_free(gpointer vq);
 G_DEFINE_AUTOPTR_CLEANUP_FUNC(VhostShadowVirtqueue, vhost_svq_free);
diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
index 297505f..ec38189 100644
--- a/hw/virtio/vhost-vdpa.c
+++ b/hw/virtio/vhost-vdpa.c
@@ -209,6 +209,21 @@ static void vhost_vdpa_listener_region_add(MemoryListe=
ner *listener,
                                          vaddr, section->readonly);
=20
     llsize =3D int128_sub(llend, int128_make64(iova));
+    if (v->shadow_vqs_enabled) {
+        DMAMap mem_region =3D {
+            .translated_addr =3D (hwaddr)(uintptr_t)vaddr,
+            .size =3D int128_get64(llsize) - 1,
+            .perm =3D IOMMU_ACCESS_FLAG(true, section->readonly),
+        };
+
+        int r =3D vhost_iova_tree_map_alloc(v->iova_tree, &mem_region);
+        if (unlikely(r !=3D IOVA_OK)) {
+            error_report("Can't allocate a mapping (%d)", r);
+            goto fail;
+        }
+
+        iova =3D mem_region.iova;
+    }
=20
     vhost_vdpa_iotlb_batch_begin_once(v);
     ret =3D vhost_vdpa_dma_map(v, iova, int128_get64(llsize),
@@ -261,6 +276,20 @@ static void vhost_vdpa_listener_region_del(MemoryListe=
ner *listener,
=20
     llsize =3D int128_sub(llend, int128_make64(iova));
=20
+    if (v->shadow_vqs_enabled) {
+        const DMAMap *result;
+        const void *vaddr =3D memory_region_get_ram_ptr(section->mr) +
+            section->offset_within_region +
+            (iova - section->offset_within_address_space);
+        DMAMap mem_region =3D {
+            .translated_addr =3D (hwaddr)(uintptr_t)vaddr,
+            .size =3D int128_get64(llsize) - 1,
+        };
+
+        result =3D vhost_iova_tree_find_iova(v->iova_tree, &mem_region);
+        iova =3D result->iova;
+        vhost_iova_tree_remove(v->iova_tree, &mem_region);
+    }
     vhost_vdpa_iotlb_batch_begin_once(v);
     ret =3D vhost_vdpa_dma_unmap(v, iova, int128_get64(llsize));
     if (ret) {
@@ -370,7 +399,7 @@ static int vhost_vdpa_init_svq(struct vhost_dev *hdev, =
struct vhost_vdpa *v,
=20
     shadow_vqs =3D g_ptr_array_new_full(hdev->nvqs, vhost_svq_free);
     for (unsigned n =3D 0; n < hdev->nvqs; ++n) {
-        g_autoptr(VhostShadowVirtqueue) svq =3D vhost_svq_new();
+        g_autoptr(VhostShadowVirtqueue) svq =3D vhost_svq_new(v->iova_tree=
);
=20
         if (unlikely(!svq)) {
             error_setg(errp, "Cannot create svq %u", n);
@@ -807,33 +836,70 @@ static int vhost_vdpa_svq_set_fds(struct vhost_dev *d=
ev,
 /**
  * Unmap a SVQ area in the device
  */
-static bool vhost_vdpa_svq_unmap_ring(struct vhost_vdpa *v, hwaddr iova,
-                                      hwaddr size)
+static bool vhost_vdpa_svq_unmap_ring(struct vhost_vdpa *v,
+                                      const DMAMap *needle)
 {
+    const DMAMap *result =3D vhost_iova_tree_find_iova(v->iova_tree, needl=
e);
+    hwaddr size;
     int r;
=20
-    size =3D ROUND_UP(size, qemu_real_host_page_size);
-    r =3D vhost_vdpa_dma_unmap(v, iova, size);
+    if (unlikely(!result)) {
+        error_report("Unable to find SVQ address to unmap");
+        return false;
+    }
+
+    size =3D ROUND_UP(result->size, qemu_real_host_page_size);
+    r =3D vhost_vdpa_dma_unmap(v, result->iova, size);
     return r =3D=3D 0;
 }
=20
 static bool vhost_vdpa_svq_unmap_rings(struct vhost_dev *dev,
                                        const VhostShadowVirtqueue *svq)
 {
+    DMAMap needle =3D {};
     struct vhost_vdpa *v =3D dev->opaque;
     struct vhost_vring_addr svq_addr;
-    size_t device_size =3D vhost_svq_device_area_size(svq);
-    size_t driver_size =3D vhost_svq_driver_area_size(svq);
     bool ok;
=20
     vhost_svq_get_vring_addr(svq, &svq_addr);
=20
-    ok =3D vhost_vdpa_svq_unmap_ring(v, svq_addr.desc_user_addr, driver_si=
ze);
+    needle.translated_addr =3D svq_addr.desc_user_addr;
+    ok =3D vhost_vdpa_svq_unmap_ring(v, &needle);
     if (unlikely(!ok)) {
         return false;
     }
=20
-    return vhost_vdpa_svq_unmap_ring(v, svq_addr.used_user_addr, device_si=
ze);
+    needle.translated_addr =3D svq_addr.used_user_addr;
+    return vhost_vdpa_svq_unmap_ring(v, &needle);
+}
+
+/**
+ * Map the SVQ area in the device
+ *
+ * @v: Vhost-vdpa device
+ * @needle: The area to search iova
+ * @errorp: Error pointer
+ */
+static bool vhost_vdpa_svq_map_ring(struct vhost_vdpa *v, DMAMap *needle,
+                                    Error **errp)
+{
+    int r;
+
+    r =3D vhost_iova_tree_map_alloc(v->iova_tree, needle);
+    if (unlikely(r !=3D IOVA_OK)) {
+        error_setg(errp, "Cannot allocate iova (%d)", r);
+        return false;
+    }
+
+    r =3D vhost_vdpa_dma_map(v, needle->iova, needle->size + 1,
+                           (void *)(uintptr_t)needle->translated_addr,
+                           needle->perm =3D=3D IOMMU_RO);
+    if (unlikely(r !=3D 0)) {
+        error_setg_errno(errp, -r, "Cannot map region to device");
+        vhost_iova_tree_remove(v->iova_tree, needle);
+    }
+
+    return r =3D=3D 0;
 }
=20
 /**
@@ -849,28 +915,44 @@ static bool vhost_vdpa_svq_map_rings(struct vhost_dev=
 *dev,
                                      struct vhost_vring_addr *addr,
                                      Error **errp)
 {
+    DMAMap device_region, driver_region;
+    struct vhost_vring_addr svq_addr;
     struct vhost_vdpa *v =3D dev->opaque;
     size_t device_size =3D vhost_svq_device_area_size(svq);
     size_t driver_size =3D vhost_svq_driver_area_size(svq);
-    int r;
+    size_t avail_offset;
+    bool ok;
=20
     ERRP_GUARD();
-    vhost_svq_get_vring_addr(svq, addr);
+    vhost_svq_get_vring_addr(svq, &svq_addr);
=20
-    r =3D vhost_vdpa_dma_map(v, addr->desc_user_addr, driver_size,
-                           (void *)(uintptr_t)addr->desc_user_addr, true);
-    if (unlikely(r !=3D 0)) {
-        error_setg_errno(errp, -r, "Cannot create vq driver region: ");
+    driver_region =3D (DMAMap) {
+        .translated_addr =3D svq_addr.desc_user_addr,
+        .size =3D driver_size - 1,
+        .perm =3D IOMMU_RO,
+    };
+    ok =3D vhost_vdpa_svq_map_ring(v, &driver_region, errp);
+    if (unlikely(!ok)) {
+        error_prepend(errp, "Cannot create vq driver region: ");
         return false;
     }
+    addr->desc_user_addr =3D driver_region.iova;
+    avail_offset =3D svq_addr.avail_user_addr - svq_addr.desc_user_addr;
+    addr->avail_user_addr =3D driver_region.iova + avail_offset;
=20
-    r =3D vhost_vdpa_dma_map(v, addr->used_user_addr, device_size,
-                           (void *)(intptr_t)addr->used_user_addr, false);
-    if (unlikely(r !=3D 0)) {
-        error_setg_errno(errp, -r, "Cannot create vq device region: ");
+    device_region =3D (DMAMap) {
+        .translated_addr =3D svq_addr.used_user_addr,
+        .size =3D device_size - 1,
+        .perm =3D IOMMU_RW,
+    };
+    ok =3D vhost_vdpa_svq_map_ring(v, &device_region, errp);
+    if (unlikely(!ok)) {
+        error_prepend(errp, "Cannot create vq device region: ");
+        vhost_vdpa_svq_unmap_ring(v, &driver_region);
     }
+    addr->used_user_addr =3D device_region.iova;
=20
-    return r =3D=3D 0;
+    return ok;
 }
=20
 static bool vhost_vdpa_svq_setup(struct vhost_dev *dev,
diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h
index 009a9f3..ee8e939 100644
--- a/include/hw/virtio/vhost-vdpa.h
+++ b/include/hw/virtio/vhost-vdpa.h
@@ -14,6 +14,7 @@
=20
 #include <gmodule.h>
=20
+#include "hw/virtio/vhost-iova-tree.h"
 #include "hw/virtio/virtio.h"
 #include "standard-headers/linux/vhost_types.h"
=20
@@ -30,6 +31,8 @@ typedef struct vhost_vdpa {
     MemoryListener listener;
     struct vhost_vdpa_iova_range iova_range;
     bool shadow_vqs_enabled;
+    /* IOVA mapping used by the Shadow Virtqueue */
+    VhostIOVATree *iova_tree;
     GPtrArray *shadow_vqs;
     struct vhost_dev *dev;
     VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX];
--=20
2.7.4