From nobody Fri Mar 14 21:20:58 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
dkim=pass;
spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
permitted sender)
smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
dmarc=pass(p=reject dis=none) header.from=oracle.com
ARC-Seal: i=1; a=rsa-sha256; t=1736529001; cv=none;
d=zohomail.com; s=zohoarc;
b=T9j10IT5MLLab3xj3naPJExWY/wRxNKcDE0oQjHDJAkFkf1m/pmqvR+oM0Oqdd4Q4hhMx1HbgoEsvJvKtFlcchMf2MAHQH6Adq+HJWqDHdACHx0ZZb5ut/hQ9/Asj/lLJRVFSvbAbggggxG5eDrvnSs7kfiMxYmMfeS91D5mXIo=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com;
s=zohoarc;
t=1736529001;
h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To;
bh=jJG0caxPU8cnGnsYt8aHP7FSNtVuznZpAH0g/XMGUW4=;
b=RXKFUeOIGWMhBCWlWTPxSMQb99V5I39NWqTm/u+vErx6ogtI+OdZhStG/HLeUZIF1K/7v4KPystlFf9yXuVouQ/8N6TD1iRrKdyo6K4euT78P9XCUXHX211B/XrZyzRm65AkV6C42xdRaed5vFg1RuxNkdiAtvDSncthS+kLEvE=
ARC-Authentication-Results: i=1; mx.zohomail.com;
dkim=pass;
spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
permitted sender)
smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
dmarc=pass header.from=<jonah.palmer@oracle.com> (p=reject dis=none)
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
mx.zohomail.com
with SMTPS id 1736529001551591.1651786724735;
Fri, 10 Jan 2025 09:10:01 -0800 (PST)
Received: from localhost ([::1] helo=lists1p.gnu.org)
by lists.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <qemu-devel-bounces@nongnu.org>)
id 1tWIVG-0006oe-7k; Fri, 10 Jan 2025 12:08:58 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <jonah.palmer@oracle.com>)
id 1tWIVD-0006ng-5a
for qemu-devel@nongnu.org; Fri, 10 Jan 2025 12:08:55 -0500
Received: from mx0b-00069f02.pphosted.com ([205.220.177.32])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <jonah.palmer@oracle.com>)
id 1tWIVB-00055w-2O
for qemu-devel@nongnu.org; Fri, 10 Jan 2025 12:08:54 -0500
Received: from pps.filterd (m0246631.ppops.net [127.0.0.1])
by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id
50AENhuW007304;
Fri, 10 Jan 2025 17:08:49 GMT
Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com
(phxpaimrmta02.appoci.oracle.com [147.154.114.232])
by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4427jpk83w-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
Fri, 10 Jan 2025 17:08:48 +0000 (GMT)
Received: from pps.filterd
(phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])
by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2)
with ESMTP id 50AGHCDp005499; Fri, 10 Jan 2025 17:08:47 GMT
Received: from pps.reinject (localhost [127.0.0.1])
by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id
43xuecp0tx-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
Fri, 10 Jan 2025 17:08:47 +0000
Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com
(phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1])
by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 50AH1lkO031831;
Fri, 10 Jan 2025 17:08:46 GMT
Received: from jonah-ol8.us.oracle.com (dhcp-10-39-193-76.vpn.oracle.com
[10.39.193.76])
by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id
43xuecp0qc-2; Fri, 10 Jan 2025 17:08:46 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc
:content-transfer-encoding:date:from:in-reply-to:message-id
:mime-version:references:subject:to; s=corp-2023-11-20; bh=jJG0c
axPU8cnGnsYt8aHP7FSNtVuznZpAH0g/XMGUW4=; b=fWcCgGoVYDbV7240DXezM
eaOwI7NqrCxI5WRF4AQ/rG6Ql8DfHNiLrFfw1ya6yTHVM2TFnRtKdcE4RJGTSlI4
KHg4lTS2BtqD1Tb9TAiIzN30ik4hGKMdUG1L+cejFwBP5lJ27Qnpwp/XSa6LgQuE
31PBSSxfjzkQqG8tjR/oHcK9X0kRqDGtYXh/Kuqqu93phKnf4rXXSyWbOqDNb3Y/
sQfBlpC+v0oF+tFt7FNzlgdhpUWnjzVX2KMsKcXa1EwWZ0cQflDD6WsxIqChXY/Y
QRmN1DzL9mmPLqD0KYQ8lWg3622ayvbZDL1HHzchSvsfN7ynz3uMy8+pV0AeqGqP
A==
From: Jonah Palmer <jonah.palmer@oracle.com>
To: qemu-devel@nongnu.org
Cc: eperezma@redhat.com, mst@redhat.com, leiyang@redhat.com,
peterx@redhat.com,
dtatulea@nvidia.com, jasowang@redhat.com, si-wei.liu@oracle.com,
boris.ostrovsky@oracle.com, jonah.palmer@oracle.com
Subject: [RFC v3 1/5] vhost-vdpa: Decouple the IOVA allocator
Date: Fri, 10 Jan 2025 12:08:30 -0500
Message-ID: <20250110170837.2747532-2-jonah.palmer@oracle.com>
X-Mailer: git-send-email 2.43.5
In-Reply-To: <20250110170837.2747532-1-jonah.palmer@oracle.com>
References: <20250110170837.2747532-1-jonah.palmer@oracle.com>
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
X-Proofpoint-Virus-Version: vendor=baseguard
engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34
definitions=2025-01-10_07,2025-01-10_03,2024-11-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0
bulkscore=0 suspectscore=0
adultscore=0 mlxlogscore=999 phishscore=0 malwarescore=0 mlxscore=0
classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000
definitions=main-2501100134
X-Proofpoint-ORIG-GUID: 6bsGfwwxvc6R7a_LdCUx62cnRgqCihBc
X-Proofpoint-GUID: 6bsGfwwxvc6R7a_LdCUx62cnRgqCihBc
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
as permitted sender) client-ip=209.51.188.17;
envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
helo=lists.gnu.org;
Received-SPF: pass client-ip=205.220.177.32;
envelope-from=jonah.palmer@oracle.com; helo=mx0b-00069f02.pphosted.com
X-Spam_score_int: -31
X-Spam_score: -3.2
X-Spam_bar: ---
X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.369,
DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001,
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001,
RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001,
SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org
X-ZohoMail-DKIM: pass (identity @oracle.com)
X-ZM-MESSAGEID: 1736529003648116600
Content-Type: text/plain; charset="utf-8"
Decouples the IOVA allocator from the full IOVA->HVA tree to support a
SVQ IOVA->HVA tree for host-only memory mappings.
The IOVA allocator still allocates an IOVA range but instead adds this
range to an IOVA-only tree (iova_map) that keeps track of allocated IOVA
ranges for both guest & host-only memory mappings.
A new API function vhost_iova_tree_insert() is also created for adding
IOVA->HVA mappings into the SVQ IOVA->HVA tree, since the allocator is
no longer doing that.
Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>
---
hw/virtio/vhost-iova-tree.c | 35 +++++++++++++++++++++++++++++++----
hw/virtio/vhost-iova-tree.h | 1 +
hw/virtio/vhost-vdpa.c | 21 ++++++++++++++++-----
net/vhost-vdpa.c | 13 +++++++++++--
4 files changed, 59 insertions(+), 11 deletions(-)
diff --git a/hw/virtio/vhost-iova-tree.c b/hw/virtio/vhost-iova-tree.c
index 3d03395a77..b1cfd17843 100644
--- a/hw/virtio/vhost-iova-tree.c
+++ b/hw/virtio/vhost-iova-tree.c
@@ -28,12 +28,15 @@ struct VhostIOVATree {
=20
/* IOVA address to qemu memory maps. */
IOVATree *iova_taddr_map;
+
+ /* Allocated IOVA addresses */
+ IOVATree *iova_map;
};
=20
/**
- * Create a new IOVA tree
+ * Create a new VhostIOVATree
*
- * Returns the new IOVA tree
+ * Returns the new VhostIOVATree
*/
VhostIOVATree *vhost_iova_tree_new(hwaddr iova_first, hwaddr iova_last)
{
@@ -44,15 +47,17 @@ VhostIOVATree *vhost_iova_tree_new(hwaddr iova_first, h=
waddr iova_last)
tree->iova_last =3D iova_last;
=20
tree->iova_taddr_map =3D iova_tree_new();
+ tree->iova_map =3D iova_tree_new();
return tree;
}
=20
/**
- * Delete an iova tree
+ * Delete a VhostIOVATree
*/
void vhost_iova_tree_delete(VhostIOVATree *iova_tree)
{
iova_tree_destroy(iova_tree->iova_taddr_map);
+ iova_tree_destroy(iova_tree->iova_map);
g_free(iova_tree);
}
=20
@@ -94,7 +99,7 @@ int vhost_iova_tree_map_alloc(VhostIOVATree *tree, DMAMap=
*map)
}
=20
/* Allocate a node in IOVA address */
- return iova_tree_alloc_map(tree->iova_taddr_map, map, iova_first,
+ return iova_tree_alloc_map(tree->iova_map, map, iova_first,
tree->iova_last);
}
=20
@@ -107,4 +112,26 @@ int vhost_iova_tree_map_alloc(VhostIOVATree *tree, DMA=
Map *map)
void vhost_iova_tree_remove(VhostIOVATree *iova_tree, DMAMap map)
{
iova_tree_remove(iova_tree->iova_taddr_map, map);
+ iova_tree_remove(iova_tree->iova_map, map);
+}
+
+/**
+ * Insert a new mapping to the IOVA->HVA tree
+ *
+ * @tree: The VhostIOVATree
+ * @map: The IOVA->HVA mapping
+ *
+ * Returns:
+ * - IOVA_OK if the map fits in the container
+ * - IOVA_ERR_INVALID if the map does not make sense (e.g. size overflow)
+ * - IOVA_ERR_OVERLAP if the IOVA range overlaps with an existing range
+ */
+int vhost_iova_tree_insert(VhostIOVATree *iova_tree, DMAMap *map)
+{
+ if (map->translated_addr + map->size < map->translated_addr ||
+ map->perm =3D=3D IOMMU_NONE) {
+ return IOVA_ERR_INVALID;
+ }
+
+ return iova_tree_insert(iova_tree->iova_taddr_map, map);
}
diff --git a/hw/virtio/vhost-iova-tree.h b/hw/virtio/vhost-iova-tree.h
index 4adfd79ff0..8bf7b64786 100644
--- a/hw/virtio/vhost-iova-tree.h
+++ b/hw/virtio/vhost-iova-tree.h
@@ -23,5 +23,6 @@ const DMAMap *vhost_iova_tree_find_iova(const VhostIOVATr=
ee *iova_tree,
const DMAMap *map);
int vhost_iova_tree_map_alloc(VhostIOVATree *iova_tree, DMAMap *map);
void vhost_iova_tree_remove(VhostIOVATree *iova_tree, DMAMap map);
+int vhost_iova_tree_insert(VhostIOVATree *iova_tree, DMAMap *map);
=20
#endif
diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
index 3cdaa12ed5..f5803f35f4 100644
--- a/hw/virtio/vhost-vdpa.c
+++ b/hw/virtio/vhost-vdpa.c
@@ -1142,18 +1142,29 @@ static void vhost_vdpa_svq_unmap_rings(struct vhost=
_dev *dev,
*
* @v: Vhost-vdpa device
* @needle: The area to search iova
+ * @taddr: The translated address (SVQ HVA)
* @errorp: Error pointer
*/
static bool vhost_vdpa_svq_map_ring(struct vhost_vdpa *v, DMAMap *needle,
- Error **errp)
+ hwaddr taddr, Error **errp)
{
int r;
=20
+ /* Allocate an IOVA range in the IOVA tree */
r =3D vhost_iova_tree_map_alloc(v->shared->iova_tree, needle);
if (unlikely(r !=3D IOVA_OK)) {
error_setg(errp, "Cannot allocate iova (%d)", r);
return false;
}
+ needle->translated_addr =3D taddr;
+
+ /* Add IOVA->HVA mapping to the IOVA->HVA tree */
+ r =3D vhost_iova_tree_insert(v->shared->iova_tree, needle);
+ if (unlikely(r !=3D IOVA_OK)) {
+ error_setg(errp, "Cannot add SVQ vring mapping (%d)", r);
+ vhost_iova_tree_remove(v->shared->iova_tree, *needle);
+ return false;
+ }
=20
r =3D vhost_vdpa_dma_map(v->shared, v->address_space_id, needle->iova,
needle->size + 1,
@@ -1192,11 +1203,11 @@ static bool vhost_vdpa_svq_map_rings(struct vhost_d=
ev *dev,
vhost_svq_get_vring_addr(svq, &svq_addr);
=20
driver_region =3D (DMAMap) {
- .translated_addr =3D svq_addr.desc_user_addr,
.size =3D driver_size - 1,
.perm =3D IOMMU_RO,
};
- ok =3D vhost_vdpa_svq_map_ring(v, &driver_region, errp);
+ ok =3D vhost_vdpa_svq_map_ring(v, &driver_region, svq_addr.desc_user_a=
ddr,
+ errp);
if (unlikely(!ok)) {
error_prepend(errp, "Cannot create vq driver region: ");
return false;
@@ -1206,11 +1217,11 @@ static bool vhost_vdpa_svq_map_rings(struct vhost_d=
ev *dev,
addr->avail_user_addr =3D driver_region.iova + avail_offset;
=20
device_region =3D (DMAMap) {
- .translated_addr =3D svq_addr.used_user_addr,
.size =3D device_size - 1,
.perm =3D IOMMU_RW,
};
- ok =3D vhost_vdpa_svq_map_ring(v, &device_region, errp);
+ ok =3D vhost_vdpa_svq_map_ring(v, &device_region, svq_addr.used_user_a=
ddr,
+ errp);
if (unlikely(!ok)) {
error_prepend(errp, "Cannot create vq device region: ");
vhost_vdpa_svq_unmap_ring(v, driver_region.translated_addr);
diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c
index 231b45246c..1ef555e04e 100644
--- a/net/vhost-vdpa.c
+++ b/net/vhost-vdpa.c
@@ -512,14 +512,23 @@ static int vhost_vdpa_cvq_map_buf(struct vhost_vdpa *=
v, void *buf, size_t size,
DMAMap map =3D {};
int r;
=20
- map.translated_addr =3D (hwaddr)(uintptr_t)buf;
map.size =3D size - 1;
map.perm =3D write ? IOMMU_RW : IOMMU_RO,
+
+ /* Allocate an IOVA range in the IOVA tree */
r =3D vhost_iova_tree_map_alloc(v->shared->iova_tree, &map);
if (unlikely(r !=3D IOVA_OK)) {
- error_report("Cannot map injected element");
+ error_report("Cannot allocate IOVA range for injected element");
return r;
}
+ map.translated_addr =3D (hwaddr)(uintptr_t)buf;
+
+ /* Add IOVA->HVA mapping to the IOVA->HVA tree */
+ r =3D vhost_iova_tree_insert(v->shared->iova_tree, &map);
+ if (unlikely(r !=3D IOVA_OK)) {
+ error_report("Cannot map injected element into IOVA->HVA tree");
+ goto dma_map_err;
+ }
=20
r =3D vhost_vdpa_dma_map(v->shared, v->address_space_id, map.iova,
vhost_vdpa_net_cvq_cmd_page_len(), buf, !write);
--=20
2.43.5