From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635533020341765.6065440546716; Fri, 29 Oct 2021 11:43:40 -0700 (PDT) Received: from localhost ([::1]:41308 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgWqp-0002hb-Av for importer@patchew.org; Fri, 29 Oct 2021 14:43:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44758) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWjq-000078-AY for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:36:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:32699) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWjm-0000Fb-VW for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:36:25 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-167-Re7VpQ2SPD-8XyI46dWGzw-1; Fri, 29 Oct 2021 14:36:18 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D55181005E4D; Fri, 29 Oct 2021 18:36:16 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 25B915D6CF; Fri, 29 Oct 2021 18:35:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532581; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gY9Awd9YT7S8jY/EksDuIvrRzLXCwfr6jy62mTr8JGg=; b=B6kGkk1PKOH2GwBBZkjP40TgNCMiDMgBAt4lsorbng9KFYaiYMuaYloPiW7+hBPO2zNWlC Et0rJE4Ji6rPQ9pjyfHpTFMLMVCZpdV/j1lBuVliXC3LPH6rgx5bLjplehVngN6CoIWk/2 bdt8Uto3849uE0lACDHQTYKVzk8H3Bc= X-MC-Unique: Re7VpQ2SPD-8XyI46dWGzw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 01/26] util: Make some iova_tree parameters const Date: Fri, 29 Oct 2021 20:35:00 +0200 Message-Id: <20211029183525.1776416-2-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533022477100001 As qemu guidelines: Unless a pointer is used to modify the pointed-to storage, give it the "const" attribute. In the particular case of iova_tree_find it allows to enforce what is requested by its comment, since the compiler would shout in case of modifying or freeing the const-qualified returned pointer. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Peter Xu Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Juan Quintela --- include/qemu/iova-tree.h | 8 ++++---- hw/i386/intel_iommu.c | 2 +- util/iova-tree.c | 12 ++++++------ 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/include/qemu/iova-tree.h b/include/qemu/iova-tree.h index b66cf93c4b..8249edd764 100644 --- a/include/qemu/iova-tree.h +++ b/include/qemu/iova-tree.h @@ -59,7 +59,7 @@ IOVATree *iova_tree_new(void); * * Return: 0 if succeeded, or <0 if error. */ -int iova_tree_insert(IOVATree *tree, DMAMap *map); +int iova_tree_insert(IOVATree *tree, const DMAMap *map); =20 /** * iova_tree_remove: @@ -74,7 +74,7 @@ int iova_tree_insert(IOVATree *tree, DMAMap *map); * * Return: 0 if succeeded, or <0 if error. */ -int iova_tree_remove(IOVATree *tree, DMAMap *map); +int iova_tree_remove(IOVATree *tree, const DMAMap *map); =20 /** * iova_tree_find: @@ -92,7 +92,7 @@ int iova_tree_remove(IOVATree *tree, DMAMap *map); * user is responsible to make sure the pointer is valid (say, no * concurrent deletion in progress). */ -DMAMap *iova_tree_find(IOVATree *tree, DMAMap *map); +const DMAMap *iova_tree_find(const IOVATree *tree, const DMAMap *map); =20 /** * iova_tree_find_address: @@ -105,7 +105,7 @@ DMAMap *iova_tree_find(IOVATree *tree, DMAMap *map); * * Return: same as iova_tree_find(). */ -DMAMap *iova_tree_find_address(IOVATree *tree, hwaddr iova); +const DMAMap *iova_tree_find_address(const IOVATree *tree, hwaddr iova); =20 /** * iova_tree_foreach: diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 75f075547f..33a8af9191 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -1105,7 +1105,7 @@ static int vtd_page_walk_one(IOMMUTLBEvent *event, vt= d_page_walk_info *info) .translated_addr =3D entry->translated_addr, .perm =3D entry->perm, }; - DMAMap *mapped =3D iova_tree_find(as->iova_tree, &target); + const DMAMap *mapped =3D iova_tree_find(as->iova_tree, &target); =20 if (event->type =3D=3D IOMMU_NOTIFIER_UNMAP && !info->notify_unmap) { trace_vtd_page_walk_one_skip_unmap(entry->iova, entry->addr_mask); diff --git a/util/iova-tree.c b/util/iova-tree.c index 7990692cbd..23ea35b7a4 100644 --- a/util/iova-tree.c +++ b/util/iova-tree.c @@ -42,14 +42,14 @@ IOVATree *iova_tree_new(void) return iova_tree; } =20 -DMAMap *iova_tree_find(IOVATree *tree, DMAMap *map) +const DMAMap *iova_tree_find(const IOVATree *tree, const DMAMap *map) { return g_tree_lookup(tree->tree, map); } =20 -DMAMap *iova_tree_find_address(IOVATree *tree, hwaddr iova) +const DMAMap *iova_tree_find_address(const IOVATree *tree, hwaddr iova) { - DMAMap map =3D { .iova =3D iova, .size =3D 0 }; + const DMAMap map =3D { .iova =3D iova, .size =3D 0 }; =20 return iova_tree_find(tree, &map); } @@ -60,7 +60,7 @@ static inline void iova_tree_insert_internal(GTree *gtree= , DMAMap *range) g_tree_insert(gtree, range, range); } =20 -int iova_tree_insert(IOVATree *tree, DMAMap *map) +int iova_tree_insert(IOVATree *tree, const DMAMap *map) { DMAMap *new; =20 @@ -96,9 +96,9 @@ void iova_tree_foreach(IOVATree *tree, iova_tree_iterator= iterator) g_tree_foreach(tree->tree, iova_tree_traverse, iterator); } =20 -int iova_tree_remove(IOVATree *tree, DMAMap *map) +int iova_tree_remove(IOVATree *tree, const DMAMap *map) { - DMAMap *overlap; + const DMAMap *overlap; =20 while ((overlap =3D iova_tree_find(tree, map))) { g_tree_remove(tree->tree, overlap); --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 163553305461137.76753915746747; Fri, 29 Oct 2021 11:44:14 -0700 (PDT) Received: from localhost ([::1]:43468 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgWrN-0004DN-J5 for importer@patchew.org; Fri, 29 Oct 2021 14:44:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44830) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWk0-0000M9-V3 for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:36:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:50521) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWjw-0000H3-Ct for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:36:36 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-208-6_SuDu0JOEejSEax-UGHzg-1; Fri, 29 Oct 2021 14:36:30 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BC3ED8018AC; Fri, 29 Oct 2021 18:36:28 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3A8FC5D6CF; Fri, 29 Oct 2021 18:36:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532591; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DYfTRWXkURGy/5j4zB03lSM+M2H3qfnYclQpIff4Atk=; b=caFUZ/lkLh4U5qCHPH4BXIsiO827d3QxWIpuyZeqy7GyqhX2OezANzi7yEcHFpE1KZsEHC G3y6GeWnWqU1XRGIsPI4vjYNxj+MTYWJ+sqeGFeIKRxYz86cHxOI2PCwrV7gVfs5mQFwqn Blazsn/DfXHnai2fojgkDaHBh0ggfb0= X-MC-Unique: 6_SuDu0JOEejSEax-UGHzg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 02/26] vhost: Fix last queue index of devices with no cvq Date: Fri, 29 Oct 2021 20:35:01 +0200 Message-Id: <20211029183525.1776416-3-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533055152100001 The -1 assumes that all devices with no cvq have an spare vq allocated for them, but with no offer of VIRTIO_NET_F_CTRL_VQ. This may not be the case, and the device may have a pair number of queues. To fix this, just resort to the lower even number of queues. Fixes: 049eb15b5fc9 ("vhost: record the last virtqueue index for the virtio= device") Signed-off-by: Eugenio P=C3=A9rez Reviewed-by: Juan Quintela --- hw/net/vhost_net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index 0d888f29a6..edf56a597f 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -330,7 +330,7 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *= ncs, NetClientState *peer; =20 if (!cvq) { - last_index -=3D 1; + last_index &=3D ~1ULL; } =20 if (!k->set_guest_notifiers) { --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635532756135259.7245755690948; Fri, 29 Oct 2021 11:39:16 -0700 (PDT) Received: from localhost ([::1]:58648 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgWmY-0003gG-HR for importer@patchew.org; Fri, 29 Oct 2021 14:39:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44964) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWkN-00011a-Jc for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:36:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:34357) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWkE-0000Pk-A4 for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:36:58 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-588-XrbwYLqZM-e4xxna8TkS1A-1; Fri, 29 Oct 2021 14:36:46 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4D8661006AA5; Fri, 29 Oct 2021 18:36:45 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 211405D6CF; Fri, 29 Oct 2021 18:36:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532609; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tYsRufIuEvIZfrRmcflNhfgNhuDBZmXOpYVdTrM3Sv8=; b=Y3jTLgdpQGQQK9KaT5pAqWBtLZrAYmS68JZJ9jXyvKUNRh5RqrruapMygT1D4ws0HSF4L+ rKUULpk+zfIzfIjSUt6HY6gk1UCEdHJFQAhIaJhz00EAtY99g2NDh1o58KpwjbhgFW9Fob jgQfbsVTzHkDH2pYUX9riSHzkeBpisU= X-MC-Unique: XrbwYLqZM-e4xxna8TkS1A-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 03/26] virtio: Add VIRTIO_F_QUEUE_STATE Date: Fri, 29 Oct 2021 20:35:02 +0200 Message-Id: <20211029183525.1776416-4-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635532757076100001 Implementation of RFC of device state capability: https://lists.oasis-open.org/archives/virtio-comment/202012/msg00005.html With this capability, vdpa device can reset it's index so it can start consuming from guest after disabling shadow virtqueue (SVQ), with state not 0. The use case is to test SVQ with virtio-pci vdpa (vp_vdpa) with nested virtualization. Spawning a L0 qemu with a virtio-net device, use vp_vdpa driver to handle it in the guest, and then spawn a L1 qemu using that vdpa device. When L1 qemu calls device to set a new state though vdpa ioctl, vp_vdpa should set each queue state though virtio VIRTIO_PCI_COMMON_Q_AVAIL_STATE. Since this is only for testing vhost-vdpa, it's added here before of proposing to kernel code. No effort is done for checking that device can actually change its state, its layout, or if the device even supports to change state at all. These will be added in the future. Also, a modified version of vp_vdpa that allows to set these in PCI config is needed. TODO: Check for feature enabled and split in virtio pci config Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 4 +++- include/standard-headers/linux/virtio_config.h | 3 +++ include/standard-headers/linux/virtio_pci.h | 2 ++ hw/virtio/virtio-pci.c | 9 +++++++++ 5 files changed, 18 insertions(+), 1 deletion(-) diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h index 2446dcd9ae..019badbd7c 100644 --- a/hw/virtio/virtio-pci.h +++ b/hw/virtio/virtio-pci.h @@ -120,6 +120,7 @@ typedef struct VirtIOPCIQueue { uint32_t desc[2]; uint32_t avail[2]; uint32_t used[2]; + uint16_t state; } VirtIOPCIQueue; =20 struct VirtIOPCIProxy { diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index 8bab9cfb75..5fe575b8f0 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -289,7 +289,9 @@ typedef struct VirtIORNGConf VirtIORNGConf; DEFINE_PROP_BIT64("iommu_platform", _state, _field, \ VIRTIO_F_IOMMU_PLATFORM, false), \ DEFINE_PROP_BIT64("packed", _state, _field, \ - VIRTIO_F_RING_PACKED, false) + VIRTIO_F_RING_PACKED, false), \ + DEFINE_PROP_BIT64("save_restore_q_state", _state, _field, \ + VIRTIO_F_QUEUE_STATE, true) =20 hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n); bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n); diff --git a/include/standard-headers/linux/virtio_config.h b/include/stand= ard-headers/linux/virtio_config.h index 22e3a85f67..59fad3eb45 100644 --- a/include/standard-headers/linux/virtio_config.h +++ b/include/standard-headers/linux/virtio_config.h @@ -90,4 +90,7 @@ * Does the device support Single Root I/O Virtualization? */ #define VIRTIO_F_SR_IOV 37 + +/* Device support save and restore virtqueue state */ +#define VIRTIO_F_QUEUE_STATE 40 #endif /* _LINUX_VIRTIO_CONFIG_H */ diff --git a/include/standard-headers/linux/virtio_pci.h b/include/standard= -headers/linux/virtio_pci.h index db7a8e2fcb..c8d9802a87 100644 --- a/include/standard-headers/linux/virtio_pci.h +++ b/include/standard-headers/linux/virtio_pci.h @@ -164,6 +164,7 @@ struct virtio_pci_common_cfg { uint32_t queue_avail_hi; /* read-write */ uint32_t queue_used_lo; /* read-write */ uint32_t queue_used_hi; /* read-write */ + uint16_t queue_avail_state; /* read-write */ }; =20 /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */ @@ -202,6 +203,7 @@ struct virtio_pci_cfg_cap { #define VIRTIO_PCI_COMMON_Q_AVAILHI 44 #define VIRTIO_PCI_COMMON_Q_USEDLO 48 #define VIRTIO_PCI_COMMON_Q_USEDHI 52 +#define VIRTIO_PCI_COMMON_Q_AVAIL_STATE 56 =20 #endif /* VIRTIO_PCI_NO_MODERN */ =20 diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 750aa47ec1..d7bb549033 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -1244,6 +1244,9 @@ static uint64_t virtio_pci_common_read(void *opaque, = hwaddr addr, case VIRTIO_PCI_COMMON_Q_USEDHI: val =3D proxy->vqs[vdev->queue_sel].used[1]; break; + case VIRTIO_PCI_COMMON_Q_AVAIL_STATE: + val =3D virtio_queue_get_last_avail_idx(vdev, vdev->queue_sel); + break; default: val =3D 0; } @@ -1330,6 +1333,8 @@ static void virtio_pci_common_write(void *opaque, hwa= ddr addr, proxy->vqs[vdev->queue_sel].avail[0], ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << = 32 | proxy->vqs[vdev->queue_sel].used[0]); + virtio_queue_set_last_avail_idx(vdev, vdev->queue_sel, + proxy->vqs[vdev->queue_sel].state); proxy->vqs[vdev->queue_sel].enabled =3D 1; } else { virtio_error(vdev, "wrong value for queue_enable %"PRIx64, val= ); @@ -1353,6 +1358,9 @@ static void virtio_pci_common_write(void *opaque, hwa= ddr addr, case VIRTIO_PCI_COMMON_Q_USEDHI: proxy->vqs[vdev->queue_sel].used[1] =3D val; break; + case VIRTIO_PCI_COMMON_Q_AVAIL_STATE: + proxy->vqs[vdev->queue_sel].state =3D val; + break; default: break; } @@ -1951,6 +1959,7 @@ static void virtio_pci_reset(DeviceState *qdev) proxy->vqs[i].desc[0] =3D proxy->vqs[i].desc[1] =3D 0; proxy->vqs[i].avail[0] =3D proxy->vqs[i].avail[1] =3D 0; proxy->vqs[i].used[0] =3D proxy->vqs[i].used[1] =3D 0; + proxy->vqs[i].state =3D 0; } =20 if (pci_is_express(dev)) { --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635533590369435.7321308662007; Fri, 29 Oct 2021 11:53:10 -0700 (PDT) Received: from localhost ([::1]:59846 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgX01-00078v-4H for importer@patchew.org; Fri, 29 Oct 2021 14:53:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45034) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWkT-000189-9X for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28659) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWkP-0000RP-8k for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:05 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-237-5KK9FeIBPJ-WUqzacbWAZQ-1; Fri, 29 Oct 2021 14:36:56 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EB04F1006AA2; Fri, 29 Oct 2021 18:36:54 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9F3BF5F4E1; Fri, 29 Oct 2021 18:36:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532619; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vM5urdBWcv1DokhHavNNPmwIEWCx+ovypgItH9Om3tE=; b=WwqWO6dUd419RB0LMsZqOmkIYXwXA+iIpVb6+htv2d9HOfqF8cCvbYOP4uHmuD4d2rmpjj 6CrUkctYFXHLOm5joUMDCW8wY9cpL6KHAinDQFoZ92kp4YRdOrCmJHQ6Yi1QkWpn/MG4U+ D9kHEpQC4Rilz14mjd6eH6OPJ7uPaac= X-MC-Unique: 5KK9FeIBPJ-WUqzacbWAZQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 04/26] virtio-net: Honor VIRTIO_CONFIG_S_DEVICE_STOPPED Date: Fri, 29 Oct 2021 20:35:03 +0200 Message-Id: <20211029183525.1776416-5-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533591544100001 So the guest can stop and start net device. It freely implements the RFC https://lists.oasis-open.org/archives/virtio-comment/202012/msg00027.html To stop (as "pause") the device is required to migrate status and vring addresses between device and SVQ. Once the device is stopped, the driver can request avail_idx, so it can be assigned to SVQ. This is a WIP commit: as with VIRTIO_F_QUEUE_STATE, is introduced in virtio_config.h before of even proposing for the kernel, with no feature flag, and, with no checking in the device. It also needs a modified vp_vdpa driver that supports to set and retrieve status. For virtio-net with qemu device there is no need to restore avail state: Since every tx and rx operation is entirely done in BQL regarding virtio, it would be enough with restore last_avail_idx with used_idx. Doing this way test the vq state part of the rest of the series. Signed-off-by: Eugenio P=C3=A9rez --- include/standard-headers/linux/virtio_config.h | 2 ++ hw/net/virtio-net.c | 6 ++++-- hw/virtio/virtio-pci.c | 7 +++++-- 3 files changed, 11 insertions(+), 4 deletions(-) diff --git a/include/standard-headers/linux/virtio_config.h b/include/stand= ard-headers/linux/virtio_config.h index 59fad3eb45..b3f6b1365d 100644 --- a/include/standard-headers/linux/virtio_config.h +++ b/include/standard-headers/linux/virtio_config.h @@ -40,6 +40,8 @@ #define VIRTIO_CONFIG_S_DRIVER_OK 4 /* Driver has finished configuring features */ #define VIRTIO_CONFIG_S_FEATURES_OK 8 +/* Device is stopped */ +#define VIRTIO_CONFIG_S_DEVICE_STOPPED 32 /* Device entered invalid state, driver must reset it */ #define VIRTIO_CONFIG_S_NEEDS_RESET 0x40 /* We've given up on this device. */ diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index f2014d5ea0..8b7b97e42d 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -198,6 +198,7 @@ static bool virtio_net_started(VirtIONet *n, uint8_t st= atus) { VirtIODevice *vdev =3D VIRTIO_DEVICE(n); return (status & VIRTIO_CONFIG_S_DRIVER_OK) && + (!(status & VIRTIO_CONFIG_S_DEVICE_STOPPED)) && (n->status & VIRTIO_NET_S_LINK_UP) && vdev->vm_running; } =20 @@ -386,7 +387,7 @@ static void virtio_net_set_status(struct VirtIODevice *= vdev, uint8_t status) qemu_flush_queued_packets(ncs); } =20 - if (!q->tx_waiting) { + if (!q->tx_waiting && !(status & VIRTIO_CONFIG_S_DEVICE_STOPPED)) { continue; } =20 @@ -1489,7 +1490,8 @@ static bool virtio_net_can_receive(NetClientState *nc) } =20 if (!virtio_queue_ready(q->rx_vq) || - !(vdev->status & VIRTIO_CONFIG_S_DRIVER_OK)) { + !(vdev->status & VIRTIO_CONFIG_S_DRIVER_OK) || + vdev->status =3D=3D VIRTIO_CONFIG_S_DEVICE_STOPPED) { return false; } =20 diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index d7bb549033..741a2bd2fa 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -327,13 +327,15 @@ static void virtio_ioport_write(void *opaque, uint32_= t addr, uint32_t val) } break; case VIRTIO_PCI_STATUS: - if (!(val & VIRTIO_CONFIG_S_DRIVER_OK)) { + if (!(val & VIRTIO_CONFIG_S_DRIVER_OK) || + val & VIRTIO_CONFIG_S_DEVICE_STOPPED) { virtio_pci_stop_ioeventfd(proxy); } =20 virtio_set_status(vdev, val & 0xFF); =20 - if (val & VIRTIO_CONFIG_S_DRIVER_OK) { + if (val & VIRTIO_CONFIG_S_DRIVER_OK && + !(val & VIRTIO_CONFIG_S_DEVICE_STOPPED)) { virtio_pci_start_ioeventfd(proxy); } =20 @@ -1335,6 +1337,7 @@ static void virtio_pci_common_write(void *opaque, hwa= ddr addr, proxy->vqs[vdev->queue_sel].used[0]); virtio_queue_set_last_avail_idx(vdev, vdev->queue_sel, proxy->vqs[vdev->queue_sel].state); + virtio_queue_update_used_idx(vdev, vdev->queue_sel); proxy->vqs[vdev->queue_sel].enabled =3D 1; } else { virtio_error(vdev, "wrong value for queue_enable %"PRIx64, val= ); --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635532798018271.91768286277124; Fri, 29 Oct 2021 11:39:58 -0700 (PDT) Received: from localhost ([::1]:59718 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgWnF-0004PI-0l for importer@patchew.org; Fri, 29 Oct 2021 14:39:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45124) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWkg-0001dh-3n for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:23634) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWke-0000Tv-89 for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:17 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-292-4-ZFNx1OMeKVUrr5OxTbfA-1; Fri, 29 Oct 2021 14:37:12 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 27756806688; Fri, 29 Oct 2021 18:37:11 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 504385D6CF; Fri, 29 Oct 2021 18:36:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532635; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=njCNO4sG44aARsLfXZ27pz7GKj3w+8Zk8L76z4qVvxI=; b=Y+Qat3h2O37a2Ky1lZaBowEK6HjtPzYw9hRsHt+wDL0+O3xy0TNlVRV5gjafZ6X3Anwpac 36yVDPxveeEkhv4u1SWYc2LKu5bSzogaYyiQuISrm/9YJDFWXLW0UCbuX7YoGIUQcBYGm5 AHwFei9kDv4o9bXGLiTPWtXmT4BQ2kQ= X-MC-Unique: 4-ZFNx1OMeKVUrr5OxTbfA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 05/26] vhost: Add x-vhost-set-shadow-vq qmp Date: Fri, 29 Oct 2021 20:35:04 +0200 Message-Id: <20211029183525.1776416-6-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635532800167100002 Command to set shadow virtqueue mode. Signed-off-by: Eugenio P=C3=A9rez Reviewed-by: Juan Quintela --- qapi/net.json | 22 ++++++++++++++++++++++ net/vhost-vdpa.c | 6 ++++++ 2 files changed, 28 insertions(+) diff --git a/qapi/net.json b/qapi/net.json index 7fab2e7cd8..b191b6787b 100644 --- a/qapi/net.json +++ b/qapi/net.json @@ -79,6 +79,28 @@ { 'command': 'netdev_del', 'data': {'id': 'str'}, 'allow-preconfig': true } =20 +## +# @x-vhost-set-shadow-vq: +# +# Use vhost shadow virtqueue. +# +# @name: the device name of the VirtIO device +# +# @set: true to use the alternate shadow VQ notifications +# +# Returns: Always error, since SVQ is not implemented at the moment. +# +# Since: 6.2 +# +# Example: +# +# -> { "execute": "x-vhost-set-shadow-vq", +# "arguments": { "name": "virtio-net", "set": false } } +# +## +{ 'command': 'x-vhost-set-shadow-vq', 'data': {'name': 'str', 'set': 'bool= '}, + 'if': 'CONFIG_VHOST_VDPA' } + ## # @NetLegacyNicOptions: # diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 49ab322511..3b360da27d 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -18,6 +18,7 @@ #include "qemu/error-report.h" #include "qemu/option.h" #include "qapi/error.h" +#include "qapi/qapi-commands-net.h" #include #include #include @@ -301,3 +302,8 @@ err: =20 return -1; } + +void qmp_x_vhost_set_shadow_vq(const char *name, bool set, Error **errp) +{ + error_setg(errp, "Shadow virtqueue still not implemented"); +} --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635533501426662.4781357458384; Fri, 29 Oct 2021 11:51:41 -0700 (PDT) Received: from localhost ([::1]:56196 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgWya-0004ad-31 for importer@patchew.org; Fri, 29 Oct 2021 14:51:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45164) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWko-0001nQ-MU for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:48632) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWkm-0000Uj-Q1 for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:26 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-564-0_NM01F2Pjy-dlNxjY9L9A-1; Fri, 29 Oct 2021 14:37:23 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C33A3802B61; Fri, 29 Oct 2021 18:37:21 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7D1A05F4E1; Fri, 29 Oct 2021 18:37:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532644; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TRAbN/rEsM8M4b0BOYgukKKarbaAR97y54QNJ5u9CIg=; b=P7RaM0YPzLwyymBNkjYvrKT1W0qsbYnBbmVQdEPDbmvBj4ta5S+QLQEtvPfaYoV/2WfSUl P52aR9JgB+yqt0aJHIRp/c8CGjxRL0linROuqT4486/N/wkz81bmgg9S16IFws5u9lTweL d0R9CJYUX24YadEX1TbXZ6ehs+t790U= X-MC-Unique: 0_NM01F2Pjy-dlNxjY9L9A-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 06/26] vhost: Add VhostShadowVirtqueue Date: Fri, 29 Oct 2021 20:35:05 +0200 Message-Id: <20211029183525.1776416-7-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533502696100001 Vhost shadow virtqueue (SVQ) is an intermediate jump for virtqueue notifications and buffers, allowing qemu to track them. While qemu is forwarding the buffers and virtqueue changes, is able to commit the memory it's being dirtied, the same way regular qemu's VirtIO devices do. This commit only exposes basic SVQ allocation and free, so changes regarding different aspects of SVQ (notifications forwarding, buffer forwarding, starting/stopping) are more isolated and easier to bisect. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 21 ++++++++++ hw/virtio/vhost-shadow-virtqueue.c | 64 ++++++++++++++++++++++++++++++ hw/virtio/meson.build | 2 +- 3 files changed, 86 insertions(+), 1 deletion(-) create mode 100644 hw/virtio/vhost-shadow-virtqueue.h create mode 100644 hw/virtio/vhost-shadow-virtqueue.c diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h new file mode 100644 index 0000000000..27ac6388fa --- /dev/null +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -0,0 +1,21 @@ +/* + * vhost shadow virtqueue + * + * SPDX-FileCopyrightText: Red Hat, Inc. 2021 + * SPDX-FileContributor: Author: Eugenio P=C3=A9rez + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef VHOST_SHADOW_VIRTQUEUE_H +#define VHOST_SHADOW_VIRTQUEUE_H + +#include "hw/virtio/vhost.h" + +typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; + +VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx); + +void vhost_svq_free(VhostShadowVirtqueue *vq); + +#endif diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c new file mode 100644 index 0000000000..38887c3433 --- /dev/null +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -0,0 +1,64 @@ +/* + * vhost shadow virtqueue + * + * SPDX-FileCopyrightText: Red Hat, Inc. 2021 + * SPDX-FileContributor: Author: Eugenio P=C3=A9rez + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "hw/virtio/vhost-shadow-virtqueue.h" + +#include "qemu/error-report.h" +#include "qemu/event_notifier.h" + +/* Shadow virtqueue to relay notifications */ +typedef struct VhostShadowVirtqueue { + /* Shadow kick notifier, sent to vhost */ + EventNotifier hdev_kick; + /* Shadow call notifier, sent to vhost */ + EventNotifier hdev_call; +} VhostShadowVirtqueue; + +/* + * Creates vhost shadow virtqueue, and instruct vhost device to use the sh= adow + * methods and file descriptors. + */ +VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx) +{ + g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); + int r; + + r =3D event_notifier_init(&svq->hdev_kick, 0); + if (r !=3D 0) { + error_report("Couldn't create kick event notifier: %s", + strerror(errno)); + goto err_init_hdev_kick; + } + + r =3D event_notifier_init(&svq->hdev_call, 0); + if (r !=3D 0) { + error_report("Couldn't create call event notifier: %s", + strerror(errno)); + goto err_init_hdev_call; + } + + return g_steal_pointer(&svq); + +err_init_hdev_call: + event_notifier_cleanup(&svq->hdev_kick); + +err_init_hdev_kick: + return NULL; +} + +/* + * Free the resources of the shadow virtqueue. + */ +void vhost_svq_free(VhostShadowVirtqueue *vq) +{ + event_notifier_cleanup(&vq->hdev_kick); + event_notifier_cleanup(&vq->hdev_call); + g_free(vq); +} diff --git a/hw/virtio/meson.build b/hw/virtio/meson.build index 521f7d64a8..2dc87613bc 100644 --- a/hw/virtio/meson.build +++ b/hw/virtio/meson.build @@ -11,7 +11,7 @@ softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost-= stub.c')) =20 virtio_ss =3D ss.source_set() virtio_ss.add(files('virtio.c')) -virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backe= nd.c')) +virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backe= nd.c', 'vhost-shadow-virtqueue.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER', if_true: files('vhost-user.c')) virtio_ss.add(when: 'CONFIG_VHOST_VDPA', if_true: files('vhost-vdpa.c')) virtio_ss.add(when: 'CONFIG_VIRTIO_BALLOON', if_true: files('virtio-balloo= n.c')) --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635532969723838.906936839674; Fri, 29 Oct 2021 11:42:49 -0700 (PDT) Received: from localhost ([::1]:38964 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgWq0-000184-9E for importer@patchew.org; Fri, 29 Oct 2021 14:42:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45198) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWkv-0001pw-30 for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45022) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWkt-0000VS-Nv for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:32 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-403-QknqlEHPOHe0009kYawdAA-1; Fri, 29 Oct 2021 14:37:28 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 778145074C; Fri, 29 Oct 2021 18:37:26 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0ADFA5D6CF; Fri, 29 Oct 2021 18:37:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532651; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=49ruMDQfquTI4OYe+3DucdxBHQ4ndcQFttXdGb+8pS0=; b=K0iUiimJCktIAjWPVhrDNy5ihdZFcseDzDQb/GvqPUs9YkPpOXppiPTNfSORyxMFJH4J0n IEQ5YBHfxueY7qcPoygn5hPINMgqiUag4cPSIEscpsJBxLm47/H5t7306PN8BrJvHx6d57 cHfFZoVl5lVHZ9cxjdDVkEhCR/9P9Yc= X-MC-Unique: QknqlEHPOHe0009kYawdAA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 07/26] vdpa: Save kick_fd in vhost-vdpa Date: Fri, 29 Oct 2021 20:35:06 +0200 Message-Id: <20211029183525.1776416-8-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635532970337100001 We need to know it to switch to Shadow VirtQueue and back to normal operation. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-vdpa.h | 1 + hw/virtio/vhost-vdpa.c | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 3ce79a646d..c79a21c3c8 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -28,6 +28,7 @@ typedef struct vhost_vdpa { MemoryListener listener; struct vhost_vdpa_iova_range iova_range; struct vhost_dev *dev; + int kick_fd[VIRTIO_QUEUE_MAX]; VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX]; } VhostVDPA; =20 diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 12661fd5b1..e6ee227385 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -702,7 +702,12 @@ static int vhost_vdpa_get_vring_base(struct vhost_dev = *dev, static int vhost_vdpa_set_vring_kick(struct vhost_dev *dev, struct vhost_vring_file *file) { + struct vhost_vdpa *v =3D dev->opaque; + int vdpa_idx =3D vhost_vdpa_get_vq_index(dev, file->index); + trace_vhost_vdpa_set_vring_kick(dev, file->index, file->fd); + + v->kick_fd[vdpa_idx] =3D file->fd; return vhost_vdpa_call(dev, VHOST_SET_VRING_KICK, file); } =20 --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635532982253722.9676280525074; Fri, 29 Oct 2021 11:43:02 -0700 (PDT) Received: from localhost ([::1]:40198 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgWqD-0001yJ-84 for importer@patchew.org; Fri, 29 Oct 2021 14:43:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45242) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWl4-0002Ge-4B for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:42 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45249) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWl2-0000XQ-74 for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:41 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-575-oCmGLfQnPWS2d_XZ62XRDQ-1; Fri, 29 Oct 2021 14:37:36 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 02F6210A8E01; Fri, 29 Oct 2021 18:37:35 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id CF4295F4E1; Fri, 29 Oct 2021 18:37:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532659; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PvByzVppnUAforFeitVo+2tsZhsvNKNgZfM5fUyV6LE=; b=iLCt5eHaqpYJqp3TB8HqZs8wxhAbge+qRD5jVN/GRF6bXO2RloF8QBXWsChv/T7jbI1r7W zkxkc3InuHJ/yKMzXdv2b22id+k38fr4zM8T5oMzdsszXe3lRVXyYNIHosTFPWyQBxlIiy Xej77vG/FMskDoUlCRQL4HIihUjkJK4= X-MC-Unique: oCmGLfQnPWS2d_XZ62XRDQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 08/26] vdpa: Add vhost_svq_get_dev_kick_notifier Date: Fri, 29 Oct 2021 20:35:07 +0200 Message-Id: <20211029183525.1776416-9-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635532983478100001 Is needed so vhost-vdpa knows what to send to device as kick event fd. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 4 ++++ hw/virtio/vhost-shadow-virtqueue.c | 10 +++++++++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 27ac6388fa..50ebddbbb9 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -11,9 +11,13 @@ #define VHOST_SHADOW_VIRTQUEUE_H =20 #include "hw/virtio/vhost.h" +#include "qemu/event_notifier.h" =20 typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 +const EventNotifier *vhost_svq_get_dev_kick_notifier( + const VhostShadowVirtqueue *= svq); + VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx); =20 void vhost_svq_free(VhostShadowVirtqueue *vq); diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 38887c3433..076418556d 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -11,7 +11,6 @@ #include "hw/virtio/vhost-shadow-virtqueue.h" =20 #include "qemu/error-report.h" -#include "qemu/event_notifier.h" =20 /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { @@ -21,6 +20,15 @@ typedef struct VhostShadowVirtqueue { EventNotifier hdev_call; } VhostShadowVirtqueue; =20 +/** + * The notifier that SVQ will use to notify the device. + */ +const EventNotifier *vhost_svq_get_dev_kick_notifier( + const VhostShadowVirtqueue = *svq) +{ + return &svq->hdev_kick; +} + /* * Creates vhost shadow virtqueue, and instruct vhost device to use the sh= adow * methods and file descriptors. --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635533767309917.5041982495109; Fri, 29 Oct 2021 11:56:07 -0700 (PDT) Received: from localhost ([::1]:36446 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgX2s-0001yW-B7 for importer@patchew.org; Fri, 29 Oct 2021 14:56:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45330) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWlH-0002j2-7V for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:46689) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWlF-0000Zf-Q5 for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:37:54 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-103-WKhs9jBnM8m6wkUeStBzrw-1; Fri, 29 Oct 2021 14:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2AC9E18125C3; Fri, 29 Oct 2021 18:37:48 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5DA9D5D6CF; Fri, 29 Oct 2021 18:37:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532673; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6Uv3dGNIBRYRCUYoeDo3vlZ3X5y8gZu5m2RYc4WKv/4=; b=V4lULdnxSHV6XIAMLl+z6jrXGwQ5OYHvlWRVClo4/g39tBfPerJrDihMDxERHpStOVA2m9 7/omgkrowHxW1DEmrLNeAkbFLICDRxktk2zTU9DK3PuqgJp+c4fM7l69cipDyTJjOMB5cN 6f2dUIv/FlfrKuxpwLDTheD3ZUVobYg= X-MC-Unique: WKhs9jBnM8m6wkUeStBzrw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 09/26] vdpa: Add vhost_svq_set_svq_kick_fd Date: Fri, 29 Oct 2021 20:35:08 +0200 Message-Id: <20211029183525.1776416-10-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533769300100001 This function allow vhost-vdpa backend to override kick_fd. There are a few pieces still missing, like the guest's kick handler in SVQ, and how to handle the first set of kick file descriptor, that has its own complexities. These will be added in next patches. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 36 ++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 50ebddbbb9..a19eede089 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -15,6 +15,7 @@ =20 typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 +void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); const EventNotifier *vhost_svq_get_dev_kick_notifier( const VhostShadowVirtqueue *= svq); =20 diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 076418556d..513d7f2782 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -11,6 +11,7 @@ #include "hw/virtio/vhost-shadow-virtqueue.h" =20 #include "qemu/error-report.h" +#include "qemu/main-loop.h" =20 /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { @@ -18,6 +19,16 @@ typedef struct VhostShadowVirtqueue { EventNotifier hdev_kick; /* Shadow call notifier, sent to vhost */ EventNotifier hdev_call; + + /* + * Borrowed virtqueue's guest to host notifier. + * To borrow it in this event notifier allows to register on the event + * loop and access the associated shadow virtqueue easily. If we use t= he + * VirtQueue, we don't have an easy way to retrieve it. + * + * So shadow virtqueue must not clean it, or we would lose VirtQueue o= ne. + */ + EventNotifier svq_kick; } VhostShadowVirtqueue; =20 /** @@ -29,6 +40,31 @@ const EventNotifier *vhost_svq_get_dev_kick_notifier( return &svq->hdev_kick; } =20 +/** + * Set a new file descriptor for the guest to kick SVQ and notify for avail + * + * @svq The svq + * @svq_kick_fd The new svq kick fd + */ +void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd) +{ + EventNotifier tmp; + + event_notifier_set_handler(&svq->svq_kick, NULL); + event_notifier_init_fd(&tmp, event_notifier_get_fd(&svq->svq_kick)); + + /* + * event_notifier_set_handler already checks for guest's notifications= if + * they arrive to the new file descriptor in the switch, so there is no + * need to explicitely check for them. + */ + event_notifier_init_fd(&svq->svq_kick, svq_kick_fd); + + if (event_notifier_test_and_clear(&tmp)) { + event_notifier_set(&svq->hdev_kick); + } +} + /* * Creates vhost shadow virtqueue, and instruct vhost device to use the sh= adow * methods and file descriptors. --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635533834461417.8728162795077; Fri, 29 Oct 2021 11:57:14 -0700 (PDT) Received: from localhost ([::1]:40338 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgX3x-0004jF-FN for importer@patchew.org; Fri, 29 Oct 2021 14:57:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45390) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWlP-0002nA-S0 for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:38:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21052) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWlN-0000aZ-KF for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:38:03 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-559-6zXcfm6BMay2s2pYq09pow-1; Fri, 29 Oct 2021 14:37:57 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 18E29802B61; Fri, 29 Oct 2021 18:37:56 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 850356F920; Fri, 29 Oct 2021 18:37:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532680; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WIcyrCQVvwgl5MGeMEzzqUJtsw1kh5CpL74HSDWWkSc=; b=M6X0iLsksAc9V/lF+yqrQUBXBry1SP5mGb9fJ4LziwOgOwE5Jqo/Yy80P6/3qCip/eX3Hg epDnNrmVXLT9b0iWPdgP05kuwsG3ggD9jflfm82wlV76sD0er7Ao85N2vql2SINhuSu9Mj wXhrB5AqOo9uw7N2kTFbAwTv7kjgXfo= X-MC-Unique: 6zXcfm6BMay2s2pYq09pow-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 10/26] vhost: Add Shadow VirtQueue kick forwarding capabilities Date: Fri, 29 Oct 2021 20:35:09 +0200 Message-Id: <20211029183525.1776416-11-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533835503100001 At this mode no buffer forwarding will be performed in SVQ mode: Qemu will just forward the guest's kicks to the device. Also, host notifiers must be disabled at SVQ start, and they will not start if SVQ has been enabled when device is stopped. This will be addressed in next patches. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 4 ++ hw/virtio/vhost-shadow-virtqueue.c | 77 +++++++++++++++++++++++++++--- 2 files changed, 74 insertions(+), 7 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index a19eede089..30ab9643b9 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -18,6 +18,10 @@ typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); const EventNotifier *vhost_svq_get_dev_kick_notifier( const VhostShadowVirtqueue *= svq); +void vhost_svq_start(struct vhost_dev *dev, unsigned idx, + VhostShadowVirtqueue *svq, int svq_kick_fd); +void vhost_svq_stop(struct vhost_dev *dev, unsigned idx, + VhostShadowVirtqueue *svq); =20 VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx); =20 diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 513d7f2782..fda60d11db 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -40,18 +40,36 @@ const EventNotifier *vhost_svq_get_dev_kick_notifier( return &svq->hdev_kick; } =20 +/* Forward guest notifications */ +static void vhost_handle_guest_kick(EventNotifier *n) +{ + VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, + svq_kick); + + if (unlikely(!event_notifier_test_and_clear(n))) { + return; + } + + event_notifier_set(&svq->hdev_kick); +} + /** - * Set a new file descriptor for the guest to kick SVQ and notify for avail + * Convenience function to set guest to SVQ kick fd * - * @svq The svq - * @svq_kick_fd The new svq kick fd + * @svq The shadow VirtQueue + * @svq_kick_fd The guest to SVQ kick fd + * @check_old Check old file descriptor for pending notifications */ -void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd) +static void vhost_svq_set_svq_kick_fd_internal(VhostShadowVirtqueue *svq, + int svq_kick_fd, + bool check_old) { EventNotifier tmp; =20 - event_notifier_set_handler(&svq->svq_kick, NULL); - event_notifier_init_fd(&tmp, event_notifier_get_fd(&svq->svq_kick)); + if (check_old) { + event_notifier_set_handler(&svq->svq_kick, NULL); + event_notifier_init_fd(&tmp, event_notifier_get_fd(&svq->svq_kick)= ); + } =20 /* * event_notifier_set_handler already checks for guest's notifications= if @@ -59,12 +77,57 @@ void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *sv= q, int svq_kick_fd) * need to explicitely check for them. */ event_notifier_init_fd(&svq->svq_kick, svq_kick_fd); + event_notifier_set_handler(&svq->svq_kick, vhost_handle_guest_kick); =20 - if (event_notifier_test_and_clear(&tmp)) { + /* + * !check_old means that we are starting SVQ, taking the descriptor fr= om + * vhost-vdpa device. This means that we can't trust old file descript= or + * pending notifications, since they could have been swallowed by kern= el + * vhost or paused device. So let it enabled, and qemu event loop will= call + * us to handle guest avail ring when SVQ is ready. + */ + if (!check_old || event_notifier_test_and_clear(&tmp)) { event_notifier_set(&svq->hdev_kick); } } =20 +/** + * Set a new file descriptor for the guest to kick SVQ and notify for avail + * + * @svq The svq + * @svq_kick_fd The svq kick fd + * + * Note that SVQ will never close the old file descriptor. + */ +void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd) +{ + vhost_svq_set_svq_kick_fd_internal(svq, svq_kick_fd, true); +} + +/* + * Start shadow virtqueue operation. + * @dev vhost device + * @hidx vhost virtqueue index + * @svq Shadow Virtqueue + */ +void vhost_svq_start(struct vhost_dev *dev, unsigned idx, + VhostShadowVirtqueue *svq, int svq_kick_fd) +{ + vhost_svq_set_svq_kick_fd_internal(svq, svq_kick_fd, false); +} + +/* + * Stop shadow virtqueue operation. + * @dev vhost device + * @idx vhost queue index + * @svq Shadow Virtqueue + */ +void vhost_svq_stop(struct vhost_dev *dev, unsigned idx, + VhostShadowVirtqueue *svq) +{ + event_notifier_set_handler(&svq->svq_kick, NULL); +} + /* * Creates vhost shadow virtqueue, and instruct vhost device to use the sh= adow * methods and file descriptors. --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635533239741767.6255092576068; Fri, 29 Oct 2021 11:47:19 -0700 (PDT) Received: from localhost ([::1]:48692 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgWuM-0007j5-7j for importer@patchew.org; Fri, 29 Oct 2021 14:47:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45480) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWlm-0003cG-9K for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:38:26 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:54896) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWlk-0000cm-MN for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:38:25 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-596-fLvx3xHJMraQG43m6U6oBA-1; Fri, 29 Oct 2021 14:38:20 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 90F6310A8E00; Fri, 29 Oct 2021 18:38:19 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 724255D6CF; Fri, 29 Oct 2021 18:37:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532704; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ECrl+MYM9kk55v993t9N84N4uC48KJLtXoj2UfvCB2w=; b=Ua0apm9ye430QsL0c3b3AQ5NSH/NuTHlGied0eIHk/LsnaG/C5qx5rOrA8mgJGmKMoim3d FiNUuURQgtP+KZtExnOHPku3XEl5Ty6P8dsCshgDE2IKofexBnBL4zgN6Wd9KncV5vgCLb DsosvlOtSPBmAf+VxWDyJPMJa6ffdKg= X-MC-Unique: fLvx3xHJMraQG43m6U6oBA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 11/26] vhost: Handle host notifiers in SVQ Date: Fri, 29 Oct 2021 20:35:10 +0200 Message-Id: <20211029183525.1776416-12-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533241983100001 If device supports host notifiers, this makes one jump less (kernel) to deliver SVQ notifications to it. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 2 ++ hw/virtio/vhost-shadow-virtqueue.c | 23 ++++++++++++++++++++++- 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 30ab9643b9..eb0a54f954 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -18,6 +18,8 @@ typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); const EventNotifier *vhost_svq_get_dev_kick_notifier( const VhostShadowVirtqueue *= svq); +void vhost_svq_set_host_mr_notifier(VhostShadowVirtqueue *svq, void *addr); + void vhost_svq_start(struct vhost_dev *dev, unsigned idx, VhostShadowVirtqueue *svq, int svq_kick_fd); void vhost_svq_stop(struct vhost_dev *dev, unsigned idx, diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index fda60d11db..e3dcc039b6 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -29,6 +29,12 @@ typedef struct VhostShadowVirtqueue { * So shadow virtqueue must not clean it, or we would lose VirtQueue o= ne. */ EventNotifier svq_kick; + + /* Device's host notifier memory region. NULL means no region */ + void *host_notifier_mr; + + /* Virtio queue shadowing */ + VirtQueue *vq; } VhostShadowVirtqueue; =20 /** @@ -50,7 +56,20 @@ static void vhost_handle_guest_kick(EventNotifier *n) return; } =20 - event_notifier_set(&svq->hdev_kick); + if (svq->host_notifier_mr) { + uint16_t *mr =3D svq->host_notifier_mr; + *mr =3D virtio_get_queue_index(svq->vq); + } else { + event_notifier_set(&svq->hdev_kick); + } +} + +/* + * Set the device's memory region notifier. addr =3D NULL clear it. + */ +void vhost_svq_set_host_mr_notifier(VhostShadowVirtqueue *svq, void *addr) +{ + svq->host_notifier_mr =3D addr; } =20 /** @@ -134,6 +153,7 @@ void vhost_svq_stop(struct vhost_dev *dev, unsigned idx, */ VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx) { + int vq_idx =3D dev->vq_index + idx; g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); int r; =20 @@ -151,6 +171,7 @@ VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *d= ev, int idx) goto err_init_hdev_call; } =20 + svq->vq =3D virtio_get_queue(dev->vdev, vq_idx); return g_steal_pointer(&svq); =20 err_init_hdev_call: --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635533925143635.4602588869974; Fri, 29 Oct 2021 11:58:45 -0700 (PDT) Received: from localhost ([::1]:45298 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgX5Q-0007zr-3n for importer@patchew.org; Fri, 29 Oct 2021 14:58:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45608) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWmC-0004bL-7P for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:38:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:30738) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWm9-0000gw-N7 for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:38:51 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-131-NJxm0sGcPkyZHA9fPP5y-g-1; Fri, 29 Oct 2021 14:38:45 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 01F8D10A8E06; Fri, 29 Oct 2021 18:38:44 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id E9C295D6CF; Fri, 29 Oct 2021 18:38:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532728; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K6ejgnOcL/Ooyo7TaWZcM2KXGoXIn+m4eMhazrdfuM0=; b=E19IMLgKc1MmYg6bhexCQbzln2wgIhVJIMioTjnstoNVmyy2PbX/kV5kvmV6lKWdyk4Ubu ItF+b8ermRfrhd0Hv+A0BW8XbO/63Yq6R4XMRhil24BVkNSjQAqGYwxHhxL0mJqSl/Y2tG eCSQO7AGIRSYvft7KUwl6xqtc6enZLU= X-MC-Unique: NJxm0sGcPkyZHA9fPP5y-g-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 12/26] vhost: Route guest->host notification through shadow virtqueue Date: Fri, 29 Oct 2021 20:35:11 +0200 Message-Id: <20211029183525.1776416-13-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533926959100001 At this mode no buffer forwarding will be performed in SVQ mode: Qemu just forward the guest's kicks to the device. Shadow virtqueue notifications forwarding is disabled when vhost_dev stops, so code flow follows usual cleanup. Also, host notifiers must be disabled at SVQ start, and they will not start if SVQ has been enabled when device is stopped. This is trivial to address, but it is left out for simplicity at this moment. This is an intermediate step before introduce the full SVQ mode, useful to test if the device is playing well with notifications forwarding. Signed-off-by: Eugenio P=C3=A9rez --- qapi/net.json | 5 +- include/hw/virtio/vhost-vdpa.h | 6 ++ hw/virtio/vhost-vdpa.c | 183 ++++++++++++++++++++++++++++++++- net/vhost-vdpa.c | 24 ++++- 4 files changed, 210 insertions(+), 8 deletions(-) diff --git a/qapi/net.json b/qapi/net.json index b191b6787b..fca2f6ebca 100644 --- a/qapi/net.json +++ b/qapi/net.json @@ -84,12 +84,13 @@ # # Use vhost shadow virtqueue. # +# SVQ can just forward notifications between the device and the guest at t= his +# moment. This will expand in future changes. +# # @name: the device name of the VirtIO device # # @set: true to use the alternate shadow VQ notifications # -# Returns: Always error, since SVQ is not implemented at the moment. -# # Since: 6.2 # # Example: diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index c79a21c3c8..6d60092c96 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -12,6 +12,8 @@ #ifndef HW_VIRTIO_VHOST_VDPA_H #define HW_VIRTIO_VHOST_VDPA_H =20 +#include + #include "hw/virtio/virtio.h" #include "standard-headers/linux/vhost_types.h" =20 @@ -27,9 +29,13 @@ typedef struct vhost_vdpa { bool iotlb_batch_begin_sent; MemoryListener listener; struct vhost_vdpa_iova_range iova_range; + bool shadow_vqs_enabled; + GPtrArray *shadow_vqs; struct vhost_dev *dev; int kick_fd[VIRTIO_QUEUE_MAX]; VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX]; } VhostVDPA; =20 +void vhost_vdpa_enable_svq(struct vhost_vdpa *v, bool enable, Error **errp= ); + #endif diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index e6ee227385..c388705e73 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -17,12 +17,14 @@ #include "hw/virtio/vhost.h" #include "hw/virtio/vhost-backend.h" #include "hw/virtio/virtio-net.h" +#include "hw/virtio/vhost-shadow-virtqueue.h" #include "hw/virtio/vhost-vdpa.h" #include "exec/address-spaces.h" #include "qemu/main-loop.h" #include "cpu.h" #include "trace.h" #include "qemu-common.h" +#include "qapi/error.h" =20 /* * Return one past the end of the end of section. Be careful with uint64_t @@ -326,6 +328,16 @@ static bool vhost_vdpa_one_time_request(struct vhost_d= ev *dev) return v->index !=3D 0; } =20 +/** + * Adaptor function to free shadow virtqueue through gpointer + * + * @svq The Shadow Virtqueue + */ +static void vhost_psvq_free(gpointer svq) +{ + vhost_svq_free(svq); +} + static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **er= rp) { struct vhost_vdpa *v; @@ -337,6 +349,7 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void = *opaque, Error **errp) dev->opaque =3D opaque ; v->listener =3D vhost_vdpa_memory_listener; v->msg_type =3D VHOST_IOTLB_MSG_V2; + v->shadow_vqs =3D g_ptr_array_new_full(dev->nvqs, vhost_psvq_free); =20 vhost_vdpa_get_iova_range(v); =20 @@ -361,7 +374,13 @@ static void vhost_vdpa_host_notifier_uninit(struct vho= st_dev *dev, n =3D &v->notifier[queue_index]; =20 if (n->addr) { - virtio_queue_set_host_notifier_mr(vdev, queue_index, &n->mr, false= ); + if (v->shadow_vqs_enabled) { + VhostShadowVirtqueue *svq =3D g_ptr_array_index(v->shadow_vqs, + queue_index); + vhost_svq_set_host_mr_notifier(svq, NULL); + } else { + virtio_queue_set_host_notifier_mr(vdev, queue_index, &n->mr, f= alse); + } object_unparent(OBJECT(&n->mr)); munmap(n->addr, page_size); n->addr =3D NULL; @@ -403,7 +422,12 @@ static int vhost_vdpa_host_notifier_init(struct vhost_= dev *dev, int queue_index) page_size, addr); g_free(name); =20 - if (virtio_queue_set_host_notifier_mr(vdev, queue_index, &n->mr, true)= ) { + if (v->shadow_vqs_enabled) { + VhostShadowVirtqueue *svq =3D g_ptr_array_index(v->shadow_vqs, + queue_index); + vhost_svq_set_host_mr_notifier(svq, addr); + } else if (virtio_queue_set_host_notifier_mr(vdev, queue_index, &n->mr, + true)) { munmap(addr, page_size); goto err; } @@ -432,6 +456,17 @@ err: return; } =20 +static void vhost_vdpa_svq_cleanup(struct vhost_dev *dev) +{ + struct vhost_vdpa *v =3D dev->opaque; + size_t idx; + + for (idx =3D 0; idx < v->shadow_vqs->len; ++idx) { + vhost_svq_stop(dev, idx, g_ptr_array_index(v->shadow_vqs, idx)); + } + g_ptr_array_free(v->shadow_vqs, true); +} + static int vhost_vdpa_cleanup(struct vhost_dev *dev) { struct vhost_vdpa *v; @@ -440,6 +475,7 @@ static int vhost_vdpa_cleanup(struct vhost_dev *dev) trace_vhost_vdpa_cleanup(dev, v); vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); memory_listener_unregister(&v->listener); + vhost_vdpa_svq_cleanup(dev); =20 dev->opaque =3D NULL; return 0; @@ -699,16 +735,27 @@ static int vhost_vdpa_get_vring_base(struct vhost_dev= *dev, return ret; } =20 +static int vhost_vdpa_set_vring_dev_kick(struct vhost_dev *dev, + struct vhost_vring_file *file) +{ + trace_vhost_vdpa_set_vring_kick(dev, file->index, file->fd); + return vhost_vdpa_call(dev, VHOST_SET_VRING_KICK, file); +} + static int vhost_vdpa_set_vring_kick(struct vhost_dev *dev, struct vhost_vring_file *file) { struct vhost_vdpa *v =3D dev->opaque; int vdpa_idx =3D vhost_vdpa_get_vq_index(dev, file->index); =20 - trace_vhost_vdpa_set_vring_kick(dev, file->index, file->fd); - v->kick_fd[vdpa_idx] =3D file->fd; - return vhost_vdpa_call(dev, VHOST_SET_VRING_KICK, file); + if (v->shadow_vqs_enabled) { + VhostShadowVirtqueue *svq =3D g_ptr_array_index(v->shadow_vqs, vdp= a_idx); + vhost_svq_set_svq_kick_fd(svq, file->fd); + return 0; + } else { + return vhost_vdpa_set_vring_dev_kick(dev, file); + } } =20 static int vhost_vdpa_set_vring_call(struct vhost_dev *dev, @@ -755,6 +802,132 @@ static bool vhost_vdpa_force_iommu(struct vhost_dev = *dev) return true; } =20 +/* + * Start or stop a shadow virtqueue in a vdpa device + * + * @dev Vhost device + * @idx Vhost device model queue index + * @svq_mode Shadow virtqueue mode + * @errp Error if any + * + * The function will not fall back previous values to vhost-vdpa device, s= o in + * case of a failure setting again the device properties calling this func= tion + * with the negated svq_mode is needed. + */ +static bool vhost_vdpa_svq_start_vq(struct vhost_dev *dev, unsigned idx, + bool svq_mode, Error **errp) +{ + struct vhost_vdpa *v =3D dev->opaque; + VhostShadowVirtqueue *svq =3D g_ptr_array_index(v->shadow_vqs, idx); + VhostVDPAHostNotifier *n =3D &v->notifier[idx]; + unsigned vq_index =3D idx + dev->vq_index; + struct vhost_vring_file vhost_kick_file =3D { + .index =3D vq_index, + }; + int r; + + if (svq_mode) { + const EventNotifier *vhost_kick =3D vhost_svq_get_dev_kick_notifie= r(svq); + + if (n->addr) { + r =3D virtio_queue_set_host_notifier_mr(dev->vdev, idx, &n->mr, + false); + + /* + * vhost_vdpa_host_notifier_init already validated as a proper + * host notifier memory region + */ + assert(r =3D=3D 0); + vhost_svq_set_host_mr_notifier(svq, n->addr); + } + vhost_svq_start(dev, idx, svq, v->kick_fd[idx]); + + vhost_kick_file.fd =3D event_notifier_get_fd(vhost_kick); + } else { + vhost_svq_stop(dev, idx, svq); + + if (n->addr) { + r =3D virtio_queue_set_host_notifier_mr(dev->vdev, idx, &n->mr, + true); + /* + * vhost_vdpa_host_notifier_init already validated as a proper + * host notifier memory region + */ + assert(r =3D=3D 0); + } + vhost_kick_file.fd =3D v->kick_fd[idx]; + } + + r =3D vhost_vdpa_set_vring_dev_kick(dev, &vhost_kick_file); + if (unlikely(r)) { + error_setg_errno(errp, -r, "vhost_vdpa_set_vring_kick failed"); + return false; + } + + return true; +} + +/** + * Enable or disable shadow virtqueue in a vhost vdpa device. + * + * This function is idempotent, to call it many times with the same value = for + * enable_svq will simply return success. + * + * @v Vhost vdpa device + * @enable True to set SVQ mode + * @errp Error pointer + */ +void vhost_vdpa_enable_svq(struct vhost_vdpa *v, bool enable, Error **errp) +{ + struct vhost_dev *hdev =3D v->dev; + unsigned n; + ERRP_GUARD(); + + if (enable =3D=3D v->shadow_vqs_enabled) { + return; + } + + if (enable) { + /* Allocate resources */ + assert(v->shadow_vqs->len =3D=3D 0); + for (n =3D 0; n < hdev->nvqs; ++n) { + VhostShadowVirtqueue *svq =3D vhost_svq_new(hdev, n); + bool ok; + + if (unlikely(!svq)) { + error_setg(errp, "Cannot create svq"); + enable =3D false; + goto err_svq_new; + } + g_ptr_array_add(v->shadow_vqs, svq); + + ok =3D vhost_vdpa_svq_start_vq(hdev, n, true, errp); + if (unlikely(!ok)) { + /* Free still not started svqs, and go with disable path */ + g_ptr_array_set_size(v->shadow_vqs, n); + enable =3D false; + break; + } + } + } + + v->shadow_vqs_enabled =3D enable; + + if (!enable) { + /* Disable all queues or clean up failed start */ + for (n =3D 0; n < v->shadow_vqs->len; ++n) { + vhost_vdpa_svq_start_vq(hdev, n, false, *errp ? NULL : errp); + } + + } + +err_svq_new: + if (!enable) { + /* Resources cleanup */ + g_ptr_array_set_size(v->shadow_vqs, 0); + } +} + const VhostOps vdpa_ops =3D { .backend_type =3D VHOST_BACKEND_TYPE_VDPA, .vhost_backend_init =3D vhost_vdpa_init, diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 3b360da27d..325971d8da 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -305,5 +305,27 @@ err: =20 void qmp_x_vhost_set_shadow_vq(const char *name, bool set, Error **errp) { - error_setg(errp, "Shadow virtqueue still not implemented"); + NetClientState *ncs; + int queues; + ERRP_GUARD(); + + queues =3D qemu_find_net_clients_except(name, &ncs, + NET_CLIENT_DRIVER_NIC, 1); + + if (!queues) { + error_setg(errp, "Device not found"); + } else if (ncs->info->type !=3D NET_CLIENT_DRIVER_VHOST_VDPA) { + error_setg(errp, "Device type is not vdpa"); + } else if (queues > 1) { + error_setg(errp, "Device has control virtqueue"); + } else { + VhostVDPAState *s =3D DO_UPCAST(VhostVDPAState, nc, ncs); + struct vhost_vdpa *v =3D &s->vhost_vdpa; + + vhost_vdpa_enable_svq(v, set, errp); + } + + if (*errp) { + error_prepend(errp, "Can't set shadow vq on %s: ", name); + } } --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635533477047981.0793637949853; Fri, 29 Oct 2021 11:51:17 -0700 (PDT) Received: from localhost ([::1]:55528 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgWyB-000480-WA for importer@patchew.org; Fri, 29 Oct 2021 14:51:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45772) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWmh-000520-Gj for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:39:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27502) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWmf-00015v-DC for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:39:23 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-89-fN4_W2kvOSq8caPcVEOfXQ-1; Fri, 29 Oct 2021 14:39:17 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C5575806688; Fri, 29 Oct 2021 18:39:15 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5A1E15D6CF; Fri, 29 Oct 2021 18:38:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532760; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WT/EWBZQfs43k9An2V+yAR/p0SPZu7tMVCR+eCykeRo=; b=H9GLp/N42vEx8Bf3ZkHceWNH9/47XlDXdi4lGZ0bqYyHD48a8t1Cf5RcVNSigKOwGuVAlX BhSg1tp1aBY/RfY6TNUD9n5bAvPGUKIadPwviLQ2bCkLQ0mi0XvPp5ju8GAWv33kjwftIA ySjn60IEcZMNp/KVT8myigaoA1WVvLE= X-MC-Unique: fN4_W2kvOSq8caPcVEOfXQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 13/26] Add vhost_svq_get_svq_call_notifier Date: Fri, 29 Oct 2021 20:35:12 +0200 Message-Id: <20211029183525.1776416-14-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533478058100001 This allows vhost-vdpa device to retrieve device -> svq call eventfd. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 2 ++ hw/virtio/vhost-shadow-virtqueue.c | 12 ++++++++++++ 2 files changed, 14 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index eb0a54f954..9e089edb17 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -18,6 +18,8 @@ typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); const EventNotifier *vhost_svq_get_dev_kick_notifier( const VhostShadowVirtqueue *= svq); +const EventNotifier *vhost_svq_get_svq_call_notifier( + const VhostShadowVirtqueue *= svq); void vhost_svq_set_host_mr_notifier(VhostShadowVirtqueue *svq, void *addr); =20 void vhost_svq_start(struct vhost_dev *dev, unsigned idx, diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index e3dcc039b6..7acac1be87 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -72,6 +72,18 @@ void vhost_svq_set_host_mr_notifier(VhostShadowVirtqueue= *svq, void *addr) svq->host_notifier_mr =3D addr; } =20 +/* + * Obtain the SVQ call notifier, where vhost device notifies SVQ that there + * exists pending used buffers. + * + * @svq Shadow Virtqueue + */ +const EventNotifier *vhost_svq_get_svq_call_notifier( + const VhostShadowVirtqueue = *svq) +{ + return &svq->hdev_call; +} + /** * Convenience function to set guest to SVQ kick fd * --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 163553374413351.86441179326346; Fri, 29 Oct 2021 11:55:44 -0700 (PDT) Received: from localhost ([::1]:36014 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgX2V-0001hF-0P for importer@patchew.org; Fri, 29 Oct 2021 14:55:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46180) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWoO-00070X-4f for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:41:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:34509) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWoI-0001Gq-Vy for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:41:07 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-353-WzI2n0QvMXGYri83154kng-1; Fri, 29 Oct 2021 14:40:59 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9906A8030A0; Fri, 29 Oct 2021 18:40:57 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2976C6F920; Fri, 29 Oct 2021 18:39:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532862; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xJ71lrFz0P4oP02IXNJus6aWSe+jXY8leG8qGrZiugA=; b=bbm9DkY7kOmsndxErLWPrZdjq4HpTQ9FOVGzMfY/JyWQzp+Jzh7h6ccXeLMqQNIOPPxlHb qF5VN6GgibaXG/FV9i2vYl/4ir3QY1TOOSX4Cm23n7pbU/7HyhKC7ImD+lk2EAsUXudWUh DtGmLNqUvOHYDpHtESEe/Fj93HvOVYg= X-MC-Unique: WzI2n0QvMXGYri83154kng-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 14/26] Add vhost_svq_set_guest_call_notifier Date: Fri, 29 Oct 2021 20:35:13 +0200 Message-Id: <20211029183525.1776416-15-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533745460100001 This allows vhost-vdpa device to set SVQ -> guest notifier to SVQ. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 16 ++++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 9e089edb17..607ec6e5eb 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -16,6 +16,7 @@ typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); +void vhost_svq_set_guest_call_notifier(VhostShadowVirtqueue *svq, int call= _fd); const EventNotifier *vhost_svq_get_dev_kick_notifier( const VhostShadowVirtqueue *= svq); const EventNotifier *vhost_svq_get_svq_call_notifier( diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 7acac1be87..6535eefccd 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -30,6 +30,9 @@ typedef struct VhostShadowVirtqueue { */ EventNotifier svq_kick; =20 + /* Guest's call notifier, where SVQ calls guest. */ + EventNotifier svq_call; + /* Device's host notifier memory region. NULL means no region */ void *host_notifier_mr; =20 @@ -84,6 +87,19 @@ const EventNotifier *vhost_svq_get_svq_call_notifier( return &svq->hdev_call; } =20 +/** + * Set the call notifier for the SVQ to call the guest + * + * @svq Shadow virtqueue + * @call_fd call notifier + * + * Called on BQL context. + */ +void vhost_svq_set_guest_call_notifier(VhostShadowVirtqueue *svq, int call= _fd) +{ + event_notifier_init_fd(&svq->svq_call, call_fd); +} + /** * Convenience function to set guest to SVQ kick fd * --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635533901966367.1429865667211; Fri, 29 Oct 2021 11:58:21 -0700 (PDT) Received: from localhost ([::1]:44872 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgX51-0007ic-SE for importer@patchew.org; Fri, 29 Oct 2021 14:58:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46260) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWp0-0007zZ-LY for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:41:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28357) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWoz-0001pR-2s for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:41:46 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-555-4Mk5a5siOMmVnWJh0cl0ZQ-1; Fri, 29 Oct 2021 14:41:41 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 95F061006AA2; Fri, 29 Oct 2021 18:41:39 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id F25055D6CF; Fri, 29 Oct 2021 18:40:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532904; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PyiLkHtoPR7tVeK2z9lK3114x884BsTPtJLaXQ3EtbM=; b=CWz1sUyxlTn+n8xL2oLqTVeaXfRjaMD5ydUqAV8D2aU8Z64HAX2YloiWWK+yW1zm3IY15P Ib6fdnE9gc0zshLwC7NgSLmGir1AKpbUzfi8KAxuhZqGPPJu6dJ9HhfAsLEUm1jNGz5gQm 1aiA8H1V6420rkRN2O0Kg/KVMd9R/QM= X-MC-Unique: 4Mk5a5siOMmVnWJh0cl0ZQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 15/26] vdpa: Save call_fd in vhost-vdpa Date: Fri, 29 Oct 2021 20:35:14 +0200 Message-Id: <20211029183525.1776416-16-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635533903298100001 We need to know it to switch to Shadow VirtQueue. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 6d60092c96..2f57b17208 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -33,6 +33,8 @@ typedef struct vhost_vdpa { GPtrArray *shadow_vqs; struct vhost_dev *dev; int kick_fd[VIRTIO_QUEUE_MAX]; + /* File descriptor the device uses to call VM/SVQ */ + int call_fd[VIRTIO_QUEUE_MAX]; VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX]; } VhostVDPA; =20 diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index c388705e73..64f71bd51b 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -761,7 +761,12 @@ static int vhost_vdpa_set_vring_kick(struct vhost_dev = *dev, static int vhost_vdpa_set_vring_call(struct vhost_dev *dev, struct vhost_vring_file *file) { + struct vhost_vdpa *v =3D dev->opaque; + int vdpa_idx =3D vhost_vdpa_get_vq_index(dev, file->index); + trace_vhost_vdpa_set_vring_call(dev, file->index, file->fd); + + v->call_fd[vdpa_idx] =3D file->fd; return vhost_vdpa_call(dev, VHOST_SET_VRING_CALL, file); } =20 --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635534099510657.6027339948971; Fri, 29 Oct 2021 12:01:39 -0700 (PDT) Received: from localhost ([::1]:51092 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgX8E-0003W6-B5 for importer@patchew.org; Fri, 29 Oct 2021 15:01:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46346) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWpK-0000go-1Q for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:42:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:38361) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWpI-0001u7-Hd for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:42:05 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-588-842dOc8tOpi2zVcmDFnv_w-1; Fri, 29 Oct 2021 14:42:01 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AB9C28066F9; Fri, 29 Oct 2021 18:41:59 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF3255D6CF; Fri, 29 Oct 2021 18:41:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532924; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1NCXt7DCFCorau/0NjCbZt7WxBnJRF287pNQ5VjnhGA=; b=WDX1EiV3iJZdNAI6mj8p5Q7gGmf/PRgn8kxHSlP5CyOMuEv4LO+oPvRmZEwvFuzuqKN8w/ NOgtzp11FdUh1JHot533DsLKifbPIRg1TJiuF3H+oWpXAgPgZAdDeBbS/sbzFEXLLVa5P4 CiwVSSF1F6viFH4PRW/mtZpq5FXIr3o= X-MC-Unique: 842dOc8tOpi2zVcmDFnv_w-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 16/26] vhost-vdpa: Take into account SVQ in vhost_vdpa_set_vring_call Date: Fri, 29 Oct 2021 20:35:15 +0200 Message-Id: <20211029183525.1776416-17-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635534101140100001 Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-vdpa.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 64f71bd51b..89d77f3452 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -758,16 +758,27 @@ static int vhost_vdpa_set_vring_kick(struct vhost_dev= *dev, } } =20 +static int vhost_vdpa_set_vring_dev_call(struct vhost_dev *dev, + struct vhost_vring_file *file) +{ + trace_vhost_vdpa_set_vring_call(dev, file->index, file->fd); + return vhost_vdpa_call(dev, VHOST_SET_VRING_CALL, file); +} + static int vhost_vdpa_set_vring_call(struct vhost_dev *dev, struct vhost_vring_file *file) { struct vhost_vdpa *v =3D dev->opaque; int vdpa_idx =3D vhost_vdpa_get_vq_index(dev, file->index); =20 - trace_vhost_vdpa_set_vring_call(dev, file->index, file->fd); - v->call_fd[vdpa_idx] =3D file->fd; - return vhost_vdpa_call(dev, VHOST_SET_VRING_CALL, file); + if (v->shadow_vqs_enabled) { + VhostShadowVirtqueue *svq =3D g_ptr_array_index(v->shadow_vqs, vdp= a_idx); + vhost_svq_set_guest_call_notifier(svq, file->fd); + return 0; + } else { + return vhost_vdpa_set_vring_dev_call(dev, file); + } } =20 static int vhost_vdpa_get_features(struct vhost_dev *dev, --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635534152448501.8057068028511; Fri, 29 Oct 2021 12:02:32 -0700 (PDT) Received: from localhost ([::1]:53950 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgX95-0005WA-9K for importer@patchew.org; Fri, 29 Oct 2021 15:02:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46380) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWpP-000110-IS for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:42:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:59773) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWpN-0001uS-Ij for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:42:11 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-307-FZJx_v9_NSygdBvz3R_k-w-1; Fri, 29 Oct 2021 14:42:05 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 222F61006AAC; Fri, 29 Oct 2021 18:42:04 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 104195D6CF; Fri, 29 Oct 2021 18:41:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532929; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jZHcr8bvny1z9QTEIgdeBhLuNOwXsIMjvMT3Kd1nK80=; b=H0nFp/p2D5/rZImRijsEKJG7x5xQC/a+fn0FPlUeOULLffx8oQuQAu/IIV/A3nmOg7j+UG VVvBoyCBhpwnZ2sVB2IY8IIwBx+Ef4117hyyuBFVzTXeWrYkXtfsJdwsWhjVyGePvfspdL Wc/O/ss+8O14/en8BgmgXHmR18HVToQ= X-MC-Unique: FZJx_v9_NSygdBvz3R_k-w-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 17/26] vhost: Route host->guest notification through shadow virtqueue Date: Fri, 29 Oct 2021 20:35:16 +0200 Message-Id: <20211029183525.1776416-18-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635534154424100001 This will make qemu aware of the device used buffers, allowing it to write the guest memory with its contents if needed. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 15 +++++++++++++++ hw/virtio/vhost-vdpa.c | 13 +++++++++++++ 2 files changed, 28 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 6535eefccd..77916d2fed 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -75,6 +75,19 @@ void vhost_svq_set_host_mr_notifier(VhostShadowVirtqueue= *svq, void *addr) svq->host_notifier_mr =3D addr; } =20 +/* Forward vhost notifications */ +static void vhost_svq_handle_call(EventNotifier *n) +{ + VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, + hdev_call); + + if (unlikely(!event_notifier_test_and_clear(n))) { + return; + } + + event_notifier_set(&svq->svq_call); +} + /* * Obtain the SVQ call notifier, where vhost device notifies SVQ that there * exists pending used buffers. @@ -200,6 +213,7 @@ VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *d= ev, int idx) } =20 svq->vq =3D virtio_get_queue(dev->vdev, vq_idx); + event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); return g_steal_pointer(&svq); =20 err_init_hdev_call: @@ -215,6 +229,7 @@ err_init_hdev_kick: void vhost_svq_free(VhostShadowVirtqueue *vq) { event_notifier_cleanup(&vq->hdev_kick); + event_notifier_set_handler(&vq->hdev_call, NULL); event_notifier_cleanup(&vq->hdev_call); g_free(vq); } diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 89d77f3452..c2580693b3 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -840,10 +840,14 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev = *dev, unsigned idx, struct vhost_vring_file vhost_kick_file =3D { .index =3D vq_index, }; + struct vhost_vring_file vhost_call_file =3D { + .index =3D idx + dev->vq_index, + }; int r; =20 if (svq_mode) { const EventNotifier *vhost_kick =3D vhost_svq_get_dev_kick_notifie= r(svq); + const EventNotifier *vhost_call =3D vhost_svq_get_svq_call_notifie= r(svq); =20 if (n->addr) { r =3D virtio_queue_set_host_notifier_mr(dev->vdev, idx, &n->mr, @@ -856,9 +860,12 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *= dev, unsigned idx, assert(r =3D=3D 0); vhost_svq_set_host_mr_notifier(svq, n->addr); } + + vhost_svq_set_guest_call_notifier(svq, v->call_fd[idx]); vhost_svq_start(dev, idx, svq, v->kick_fd[idx]); =20 vhost_kick_file.fd =3D event_notifier_get_fd(vhost_kick); + vhost_call_file.fd =3D event_notifier_get_fd(vhost_call); } else { vhost_svq_stop(dev, idx, svq); =20 @@ -872,6 +879,7 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *d= ev, unsigned idx, assert(r =3D=3D 0); } vhost_kick_file.fd =3D v->kick_fd[idx]; + vhost_call_file.fd =3D v->call_fd[idx]; } =20 r =3D vhost_vdpa_set_vring_dev_kick(dev, &vhost_kick_file); @@ -879,6 +887,11 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *= dev, unsigned idx, error_setg_errno(errp, -r, "vhost_vdpa_set_vring_kick failed"); return false; } + r =3D vhost_vdpa_set_vring_dev_call(dev, &vhost_call_file); + if (unlikely(r)) { + error_setg_errno(errp, -r, "vhost_vdpa_set_vring_call failed"); + return false; + } =20 return true; } --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635534376923406.418066898597; Fri, 29 Oct 2021 12:06:16 -0700 (PDT) Received: from localhost ([::1]:34814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgXCh-0003AY-Q9 for importer@patchew.org; Fri, 29 Oct 2021 15:06:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46528) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWqD-0003Bn-Hy for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:43:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:49561) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWqB-0001yx-Hl for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:43:01 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-236-e1eOPHcxMvmaa6uF8uHZ0A-1; Fri, 29 Oct 2021 14:42:55 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 326553FA1; Fri, 29 Oct 2021 18:42:54 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7BB135D6CF; Fri, 29 Oct 2021 18:42:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635532978; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NZj9KElbujbo4le7lhfi0zaoetHlo5qDh/T488RMp3A=; b=aF0GaA6GTUSi/E7w/sOZmKT3FRC4UTu1s43cEejZReyR3xzubMcKw49I1oWCSvCOo3qZR5 f5GGzxyzTN9RgF+WUzvMU+DP93pfORIX+rNHeBYlPniZXK0kEXDOnexPu+tqYZ5qZCvbxO 43wclJhFoVpEcriR3sjiz1ydxeHp5ns= X-MC-Unique: e1eOPHcxMvmaa6uF8uHZ0A-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 18/26] virtio: Add vhost_shadow_vq_get_vring_addr Date: Fri, 29 Oct 2021 20:35:17 +0200 Message-Id: <20211029183525.1776416-19-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635534378730100001 It reports the shadow virtqueue address from qemu virtual address space. Since this will be different from the guest's vaddr, but device can access it, SVQ takes special care about its alignment & lack of garbage data. It assumes that IOMMU will work in host_page_size ranges for that. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 4 +++ hw/virtio/vhost-shadow-virtqueue.c | 51 ++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 607ec6e5eb..ed647d9648 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -22,6 +22,10 @@ const EventNotifier *vhost_svq_get_dev_kick_notifier( const EventNotifier *vhost_svq_get_svq_call_notifier( const VhostShadowVirtqueue *= svq); void vhost_svq_set_host_mr_notifier(VhostShadowVirtqueue *svq, void *addr); +void vhost_svq_get_vring_addr(const VhostShadowVirtqueue *svq, + struct vhost_vring_addr *addr); +size_t vhost_svq_driver_area_size(const VhostShadowVirtqueue *svq); +size_t vhost_svq_device_area_size(const VhostShadowVirtqueue *svq); =20 void vhost_svq_start(struct vhost_dev *dev, unsigned idx, VhostShadowVirtqueue *svq, int svq_kick_fd); diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 77916d2fed..4a37ed62a8 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -9,12 +9,16 @@ =20 #include "qemu/osdep.h" #include "hw/virtio/vhost-shadow-virtqueue.h" +#include "standard-headers/linux/vhost_types.h" =20 #include "qemu/error-report.h" #include "qemu/main-loop.h" =20 /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { + /* Shadow vring */ + struct vring vring; + /* Shadow kick notifier, sent to vhost */ EventNotifier hdev_kick; /* Shadow call notifier, sent to vhost */ @@ -38,6 +42,9 @@ typedef struct VhostShadowVirtqueue { =20 /* Virtio queue shadowing */ VirtQueue *vq; + + /* Virtio device */ + VirtIODevice *vdev; } VhostShadowVirtqueue; =20 /** @@ -113,6 +120,35 @@ void vhost_svq_set_guest_call_notifier(VhostShadowVirt= queue *svq, int call_fd) event_notifier_init_fd(&svq->svq_call, call_fd); } =20 +/* + * Get the shadow vq vring address. + * @svq Shadow virtqueue + * @addr Destination to store address + */ +void vhost_svq_get_vring_addr(const VhostShadowVirtqueue *svq, + struct vhost_vring_addr *addr) +{ + addr->desc_user_addr =3D (uint64_t)svq->vring.desc; + addr->avail_user_addr =3D (uint64_t)svq->vring.avail; + addr->used_user_addr =3D (uint64_t)svq->vring.used; +} + +size_t vhost_svq_driver_area_size(const VhostShadowVirtqueue *svq) +{ + uint16_t vq_idx =3D virtio_get_queue_index(svq->vq); + size_t desc_size =3D virtio_queue_get_desc_size(svq->vdev, vq_idx); + size_t avail_size =3D virtio_queue_get_avail_size(svq->vdev, vq_idx); + + return ROUND_UP(desc_size + avail_size, qemu_real_host_page_size); +} + +size_t vhost_svq_device_area_size(const VhostShadowVirtqueue *svq) +{ + uint16_t vq_idx =3D virtio_get_queue_index(svq->vq); + size_t used_size =3D virtio_queue_get_used_size(svq->vdev, vq_idx); + return ROUND_UP(used_size, qemu_real_host_page_size); +} + /** * Convenience function to set guest to SVQ kick fd * @@ -195,6 +231,10 @@ void vhost_svq_stop(struct vhost_dev *dev, unsigned id= x, VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx) { int vq_idx =3D dev->vq_index + idx; + unsigned num =3D virtio_queue_get_num(dev->vdev, vq_idx); + size_t desc_size =3D virtio_queue_get_desc_size(dev->vdev, vq_idx); + size_t driver_size; + size_t device_size; g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); int r; =20 @@ -213,6 +253,15 @@ VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *= dev, int idx) } =20 svq->vq =3D virtio_get_queue(dev->vdev, vq_idx); + svq->vdev =3D dev->vdev; + driver_size =3D vhost_svq_driver_area_size(svq); + device_size =3D vhost_svq_device_area_size(svq); + svq->vring.num =3D num; + svq->vring.desc =3D qemu_memalign(qemu_real_host_page_size, driver_siz= e); + svq->vring.avail =3D (void *)((char *)svq->vring.desc + desc_size); + memset(svq->vring.desc, 0, driver_size); + svq->vring.used =3D qemu_memalign(qemu_real_host_page_size, device_siz= e); + memset(svq->vring.used, 0, device_size); event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); return g_steal_pointer(&svq); =20 @@ -231,5 +280,7 @@ void vhost_svq_free(VhostShadowVirtqueue *vq) event_notifier_cleanup(&vq->hdev_kick); event_notifier_set_handler(&vq->hdev_call, NULL); event_notifier_cleanup(&vq->hdev_call); + qemu_vfree(vq->vring.desc); + qemu_vfree(vq->vring.used); g_free(vq); } --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635534607175776.8604903922611; Fri, 29 Oct 2021 12:10:07 -0700 (PDT) Received: from localhost ([::1]:46920 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgXGQ-00032f-3x for importer@patchew.org; Fri, 29 Oct 2021 15:10:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrF-00050R-FK for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:59459) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrD-0002Zz-JG for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:05 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-151-yH9DDElbOMGEF0O3Oe-QJw-1; Fri, 29 Oct 2021 14:43:59 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4C520802B7A; Fri, 29 Oct 2021 18:43:58 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C2A95D6CF; Fri, 29 Oct 2021 18:42:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635533042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PMpRhEmuVv3AwePRNnnG/C1BKzevnWMVK+w+0BrrWgs=; b=GxgAGAQWD3/+cwNChJ2zUslEi82Jriaktlm0+7OAXkguW3bdX3EQwue/GGYX8UC6Mm+eAh 3Sb8WrMsttr2hCdPkCcx6eO57Q4P2ujNzsonm2av0TR48xQWRl8jp4OFlUb99PoZgOFnkp 1Hf/CwKCt66KIL4WFTVixU00mhFexPo= X-MC-Unique: yH9DDElbOMGEF0O3Oe-QJw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 19/26] vdpa: ack VIRTIO_F_QUEUE_STATE if device supports it Date: Fri, 29 Oct 2021 20:35:18 +0200 Message-Id: <20211029183525.1776416-20-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635534609455100001 This is needed to enable or disable SVQ. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-vdpa.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index c2580693b3..fc8396ba8a 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -521,6 +521,9 @@ static int vhost_vdpa_set_features(struct vhost_dev *de= v, if (vhost_vdpa_one_time_request(dev)) { return 0; } + if (dev->features & BIT_ULL(VIRTIO_F_QUEUE_STATE)) { + features |=3D BIT_ULL(VIRTIO_F_QUEUE_STATE); + } =20 trace_vhost_vdpa_set_features(dev, features); ret =3D vhost_vdpa_call(dev, VHOST_SET_FEATURES, &features); --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635534156284482.99240025098027; Fri, 29 Oct 2021 12:02:36 -0700 (PDT) Received: from localhost ([::1]:54208 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgX99-0005g9-5w for importer@patchew.org; Fri, 29 Oct 2021 15:02:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46714) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrI-00059U-JZ for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:56603) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrG-0002ja-3R for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:08 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-98-9ElmpipUM1eh7kbVVFu7QQ-1; Fri, 29 Oct 2021 14:44:04 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BB3B45074C; Fri, 29 Oct 2021 18:44:02 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id A67E95D6CF; Fri, 29 Oct 2021 18:43:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635533045; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q0/mZGXDhUBt6N133Qs8OckdZIyLUvn28K5bTFro/dE=; b=hZjMUt6oVV2ptRbCVM4VOJ8lWeneU0/fyogxPAH0nZa1lsvTkQlHKg2XTz1zOvxqauq+ra 2ivaGp8ZrKgLnTzXq3bitY0+v5qGS3Szpobb2EhEtBxUFqzMQYRsRu/c0834IBa44QLIAy 43VTaRmwf19dt3dj5NAIz8s5G0/tKFw= X-MC-Unique: 9ElmpipUM1eh7kbVVFu7QQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 20/26] vhost: Add vhost_svq_valid_device_features to shadow vq Date: Fri, 29 Oct 2021 20:35:19 +0200 Message-Id: <20211029183525.1776416-21-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635534156968100001 This allows it to test if the guest has aknowledge an invalid transport feature for SVQ. This will include packed vq layout, invalid descriptors or event idx at the moment we start forwarding buffers. We don't check for device features here since they will be re-negotiated again. This allows SVQ to both use more advanced features of the device when they are available and the guest is not capable of run them, and to make SVQ compatible with future transport features. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 2 ++ hw/virtio/vhost-shadow-virtqueue.c | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index ed647d9648..946b2c6295 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -15,6 +15,8 @@ =20 typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 +bool vhost_svq_valid_device_features(uint64_t *features); + void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); void vhost_svq_set_guest_call_notifier(VhostShadowVirtqueue *svq, int call= _fd); const EventNotifier *vhost_svq_get_dev_kick_notifier( diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 4a37ed62a8..6e0508a231 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -56,6 +56,12 @@ const EventNotifier *vhost_svq_get_dev_kick_notifier( return &svq->hdev_kick; } =20 +/* If the device is using some of these, SVQ cannot communicate */ +bool vhost_svq_valid_device_features(uint64_t *dev_features) +{ + return true; +} + /* Forward guest notifications */ static void vhost_handle_guest_kick(EventNotifier *n) { --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635534361439936.570328909286; Fri, 29 Oct 2021 12:06:01 -0700 (PDT) Received: from localhost ([::1]:59982 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgXBQ-00017m-RN for importer@patchew.org; Fri, 29 Oct 2021 15:04:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46746) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrO-0005Va-5P for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:52218) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrM-000315-9e for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:13 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-479-VSiS6s6rN3aRnDEClTT2GQ-1; Fri, 29 Oct 2021 14:44:08 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2EF325074B; Fri, 29 Oct 2021 18:44:07 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 20921641A7; Fri, 29 Oct 2021 18:44:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635533051; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qQM8e6+fu2Ajo8TPr4DoRRrfbfomdJ00aARP9SJh5Nw=; b=cIlvax+WJWjyDhOQ4xFRKZK5+dESqP09YRBaAUI5SzSQFDL9ouxB2G/Kj5aTTvGtoYytyo bTemGB/ZL0ZG4maevP1EatF0+69OGP+x0gTiUI57Hjo2PDjbXR8E19aBdNUkacA9REoWyn MAaWIDTOyXNF7THeW0PWGu0RRpxWP3w= X-MC-Unique: VSiS6s6rN3aRnDEClTT2GQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 21/26] vhost: Add vhost_svq_valid_guest_features to shadow vq Date: Fri, 29 Oct 2021 20:35:20 +0200 Message-Id: <20211029183525.1776416-22-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635534362523100001 This allows it to test if the guest has aknowledge an invalid transport feature for SVQ. This will include packed vq layout or event_idx, where VirtIO device needs help from SVQ. There is not needed at this moment, but since SVQ will not re-negotiate features again with the guest, a failure in acknowledge them is fatal for SVQ. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 946b2c6295..ac55588009 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -16,6 +16,7 @@ typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 bool vhost_svq_valid_device_features(uint64_t *features); +bool vhost_svq_valid_guest_features(uint64_t *features); =20 void vhost_svq_set_svq_kick_fd(VhostShadowVirtqueue *svq, int svq_kick_fd); void vhost_svq_set_guest_call_notifier(VhostShadowVirtqueue *svq, int call= _fd); diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 6e0508a231..cb9ffcb015 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -62,6 +62,12 @@ bool vhost_svq_valid_device_features(uint64_t *dev_featu= res) return true; } =20 +/* If the guest is using some of these, SVQ cannot communicate */ +bool vhost_svq_valid_guest_features(uint64_t *guest_features) +{ + return true; +} + /* Forward guest notifications */ static void vhost_handle_guest_kick(EventNotifier *n) { --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635534811972180.11091948731166; Fri, 29 Oct 2021 12:13:31 -0700 (PDT) Received: from localhost ([::1]:55608 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgXJi-0000aB-RQ for importer@patchew.org; Fri, 29 Oct 2021 15:13:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46768) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrS-0005oX-Ee for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45474) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrP-00031W-E0 for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:18 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-457-Gk4-UrNFOGaS_fWLgG00Cw-1; Fri, 29 Oct 2021 14:44:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CC88C18125D1; Fri, 29 Oct 2021 18:44:11 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8AC0C5F4E1; Fri, 29 Oct 2021 18:44:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635533054; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f5rA2YSg8fVGdOk1GRK/mLZSUPgPWmKXUwr9s5PeuVc=; b=TAYcHr4hcrdAbxD+LapFGvwQytLAcO0yX/k58jBx+kdNhJlgt4eyhhI3bygAs0EK6hmd9W lBhVHmLAPyQX60md98YzegCU058R2vASG4+Fhr5ZgBmHYPRjSNo0Q0esRbw5CeZ0qCTA7V EHfQ584f2BfCXCXck2IOzMIfbsma4M0= X-MC-Unique: Gk4-UrNFOGaS_fWLgG00Cw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 22/26] vhost: Shadow virtqueue buffers forwarding Date: Fri, 29 Oct 2021 20:35:21 +0200 Message-Id: <20211029183525.1776416-23-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635534813745100001 Initial version of shadow virtqueue that actually forward buffers. There are no iommu support at the moment, and that will be addressed in future patches of this series. Since all vhost-vdpa devices uses forced IOMMU, this means that SVQ is not usable at this point of the series on any device. For simplicity it only supports modern devices, that expects vring in little endian, with split ring and no event idx or indirect descriptors. Support for them will not be added in this series. It reuses the VirtQueue code for the device part. The driver part is based on Linux's virtio_ring driver, but with stripped functionality and optimizations so it's easier to review. Later commits add simpler ones. However to forwarding buffers have some particular pieces: One of the most unexpected ones is that a guest's buffer can expand through more than one descriptor in SVQ. While this is handled gracefully by qemu's emulated virtio devices, it may cause unexpected SVQ queue full. This patch also solves it checking for this condition at both guest's kicks and device's calls. The code may be more elegant in the future if SVQ code runs in its own iocontext. Note that vhost_vdpa_get_vq_state trust the device to write its status to used_idx at pause(), finishing all in-flight descriptors. This may not be enough for complex devices, but other development like usage of inflight_fd on top of this solution may extend the usage in the future. In particular, SVQ trust it to recover guest's virtqueue at start, and to mark as used the latest descriptors used by the device in the meantime. Signed-off-by: Eugenio P=C3=A9rez --- qapi/net.json | 5 +- hw/virtio/vhost-shadow-virtqueue.c | 400 +++++++++++++++++++++++++++-- hw/virtio/vhost-vdpa.c | 144 ++++++++++- 3 files changed, 521 insertions(+), 28 deletions(-) diff --git a/qapi/net.json b/qapi/net.json index fca2f6ebca..1c6d3b2179 100644 --- a/qapi/net.json +++ b/qapi/net.json @@ -84,12 +84,9 @@ # # Use vhost shadow virtqueue. # -# SVQ can just forward notifications between the device and the guest at t= his -# moment. This will expand in future changes. -# # @name: the device name of the VirtIO device # -# @set: true to use the alternate shadow VQ notifications +# @set: true to use the alternate shadow VQ # # Since: 6.2 # diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index cb9ffcb015..ad1b2342be 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -9,6 +9,9 @@ =20 #include "qemu/osdep.h" #include "hw/virtio/vhost-shadow-virtqueue.h" +#include "hw/virtio/vhost.h" +#include "hw/virtio/virtio-access.h" + #include "standard-headers/linux/vhost_types.h" =20 #include "qemu/error-report.h" @@ -45,6 +48,27 @@ typedef struct VhostShadowVirtqueue { =20 /* Virtio device */ VirtIODevice *vdev; + + /* Map for returning guest's descriptors */ + VirtQueueElement **ring_id_maps; + + /* Next VirtQueue element that guest made available */ + VirtQueueElement *next_guest_avail_elem; + + /* Next head to expose to device */ + uint16_t avail_idx_shadow; + + /* Next free descriptor */ + uint16_t free_head; + + /* Last seen used idx */ + uint16_t shadow_used_idx; + + /* Next head to consume from device */ + uint16_t last_used_idx; + + /* Cache for the exposed notification flag */ + bool notification; } VhostShadowVirtqueue; =20 /** @@ -56,25 +80,174 @@ const EventNotifier *vhost_svq_get_dev_kick_notifier( return &svq->hdev_kick; } =20 -/* If the device is using some of these, SVQ cannot communicate */ +/** + * VirtIO transport device feature acknowledge + * + * @dev_features The device features. If success, the acknowledged featur= es. + * + * Returns true if SVQ can go with a subset of these, false otherwise. + */ bool vhost_svq_valid_device_features(uint64_t *dev_features) { - return true; + uint64_t b; + bool r =3D true; + + for (b =3D VIRTIO_TRANSPORT_F_START; b <=3D VIRTIO_TRANSPORT_F_END; ++= b) { + switch (b) { + case VIRTIO_F_NOTIFY_ON_EMPTY: + case VIRTIO_F_ANY_LAYOUT: + continue; + + case VIRTIO_F_ACCESS_PLATFORM: + /* SVQ does not know how to translate addresses */ + if (*dev_features & BIT_ULL(b)) { + clear_bit(b, dev_features); + r =3D false; + } + break; + + case VIRTIO_F_VERSION_1: + /* SVQ trust that guest vring is little endian */ + if (!(*dev_features & BIT_ULL(b))) { + set_bit(b, dev_features); + r =3D false; + } + continue; + + default: + if (*dev_features & BIT_ULL(b)) { + clear_bit(b, dev_features); + } + } + } + + return r; } =20 -/* If the guest is using some of these, SVQ cannot communicate */ +/** + * Check of guest's acknowledge features. + * + * @guest_features The guest's acknowledged features + * + * Returns true if SVQ can handle them, false otherwise. + */ bool vhost_svq_valid_guest_features(uint64_t *guest_features) { - return true; + static const uint64_t transport =3D MAKE_64BIT_MASK(VIRTIO_TRANSPORT_F= _START, + VIRTIO_TRANSPORT_F_END - VIRTIO_TRANSPORT_F_ST= ART); + + /* These transport features are handled by VirtQueue */ + static const uint64_t valid =3D (BIT_ULL(VIRTIO_RING_F_INDIRECT_DESC) | + BIT_ULL(VIRTIO_F_VERSION_1)); + + /* We are only interested in transport-related feature bits */ + uint64_t guest_transport_features =3D (*guest_features) & transport; + + *guest_features &=3D (valid | ~transport); + return !(guest_transport_features & (transport ^ valid)); } =20 -/* Forward guest notifications */ -static void vhost_handle_guest_kick(EventNotifier *n) +/** + * Number of descriptors that SVQ can make available from the guest. + * + * @svq The svq + */ +static uint16_t vhost_svq_available_slots(const VhostShadowVirtqueue *svq) { - VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, - svq_kick); + return svq->vring.num - (svq->avail_idx_shadow - svq->shadow_used_idx); +} =20 - if (unlikely(!event_notifier_test_and_clear(n))) { +static void vhost_svq_set_notification(VhostShadowVirtqueue *svq, bool ena= ble) +{ + uint16_t notification_flag; + + if (svq->notification =3D=3D enable) { + return; + } + + notification_flag =3D cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + + svq->notification =3D enable; + if (enable) { + svq->vring.avail->flags &=3D ~notification_flag; + } else { + svq->vring.avail->flags |=3D notification_flag; + } +} + +static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, + const struct iovec *iovec, + size_t num, bool more_descs, bool writ= e) +{ + uint16_t i =3D svq->free_head, last =3D svq->free_head; + unsigned n; + uint16_t flags =3D write ? cpu_to_le16(VRING_DESC_F_WRITE) : 0; + vring_desc_t *descs =3D svq->vring.desc; + + if (num =3D=3D 0) { + return; + } + + for (n =3D 0; n < num; n++) { + if (more_descs || (n + 1 < num)) { + descs[i].flags =3D flags | cpu_to_le16(VRING_DESC_F_NEXT); + } else { + descs[i].flags =3D flags; + } + descs[i].addr =3D cpu_to_le64((hwaddr)iovec[n].iov_base); + descs[i].len =3D cpu_to_le32(iovec[n].iov_len); + + last =3D i; + i =3D cpu_to_le16(descs[i].next); + } + + svq->free_head =3D le16_to_cpu(descs[last].next); +} + +static unsigned vhost_svq_add_split(VhostShadowVirtqueue *svq, + VirtQueueElement *elem) +{ + int head; + unsigned avail_idx; + vring_avail_t *avail =3D svq->vring.avail; + + head =3D svq->free_head; + + /* We need some descriptors here */ + assert(elem->out_num || elem->in_num); + + vhost_vring_write_descs(svq, elem->out_sg, elem->out_num, + elem->in_num > 0, false); + vhost_vring_write_descs(svq, elem->in_sg, elem->in_num, false, true); + + /* + * Put entry in available array (but don't update avail->idx until they + * do sync). + */ + avail_idx =3D svq->avail_idx_shadow & (svq->vring.num - 1); + avail->ring[avail_idx] =3D cpu_to_le16(head); + svq->avail_idx_shadow++; + + /* Update avail index after the descriptor is wrote */ + smp_wmb(); + avail->idx =3D cpu_to_le16(svq->avail_idx_shadow); + + return head; + +} + +static void vhost_svq_add(VhostShadowVirtqueue *svq, VirtQueueElement *ele= m) +{ + unsigned qemu_head =3D vhost_svq_add_split(svq, elem); + + svq->ring_id_maps[qemu_head] =3D elem; +} + +static void vhost_svq_kick(VhostShadowVirtqueue *svq) +{ + /* We need to expose available array entries before checking used flag= s */ + smp_mb(); + if (svq->vring.used->flags & VRING_USED_F_NO_NOTIFY) { return; } =20 @@ -86,25 +259,188 @@ static void vhost_handle_guest_kick(EventNotifier *n) } } =20 -/* - * Set the device's memory region notifier. addr =3D NULL clear it. +/** + * Forward available buffers. + * + * @svq Shadow VirtQueue + * + * Note that this function does not guarantee that all guest's available + * buffers are available to the device in SVQ avail ring. The guest may ha= ve + * exposed a GPA / GIOVA congiuous buffer, but it may not be contiguous in= qemu + * vaddr. + * + * If that happens, guest's kick notifications will be disabled until devi= ce + * makes some buffers used. */ -void vhost_svq_set_host_mr_notifier(VhostShadowVirtqueue *svq, void *addr) +static void vhost_handle_guest_kick(VhostShadowVirtqueue *svq) { - svq->host_notifier_mr =3D addr; + /* Clear event notifier */ + event_notifier_test_and_clear(&svq->svq_kick); + + /* Make available as many buffers as possible */ + do { + if (virtio_queue_get_notification(svq->vq)) { + virtio_queue_set_notification(svq->vq, false); + } + + while (true) { + VirtQueueElement *elem; + + if (svq->next_guest_avail_elem) { + elem =3D g_steal_pointer(&svq->next_guest_avail_elem); + } else { + elem =3D virtqueue_pop(svq->vq, sizeof(*elem)); + } + + if (!elem) { + break; + } + + if (elem->out_num + elem->in_num > + vhost_svq_available_slots(svq)) { + /* + * This condition is possible since a contiguous buffer in= GPA + * does not imply a contiguous buffer in qemu's VA + * scatter-gather segments. If that happen, the buffer exp= osed + * to the device needs to be a chain of descriptors at this + * moment. + * + * SVQ cannot hold more available buffers if we are here: + * queue the current guest descriptor and ignore further k= icks + * until some elements are used. + */ + svq->next_guest_avail_elem =3D elem; + return; + } + + vhost_svq_add(svq, elem); + vhost_svq_kick(svq); + } + + virtio_queue_set_notification(svq->vq, true); + } while (!virtio_queue_empty(svq->vq)); +} + +/** + * Handle guest's kick. + * + * @n guest kick event notifier, the one that guest set to notify svq. + */ +static void vhost_handle_guest_kick_notifier(EventNotifier *n) +{ + VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, + svq_kick); + vhost_handle_guest_kick(svq); +} + +static bool vhost_svq_more_used(VhostShadowVirtqueue *svq) +{ + if (svq->last_used_idx !=3D svq->shadow_used_idx) { + return true; + } + + svq->shadow_used_idx =3D cpu_to_le16(svq->vring.used->idx); + + return svq->last_used_idx !=3D svq->shadow_used_idx; +} + +static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq) +{ + vring_desc_t *descs =3D svq->vring.desc; + const vring_used_t *used =3D svq->vring.used; + vring_used_elem_t used_elem; + uint16_t last_used; + + if (!vhost_svq_more_used(svq)) { + return NULL; + } + + /* Only get used array entries after they have been exposed by dev */ + smp_rmb(); + last_used =3D svq->last_used_idx & (svq->vring.num - 1); + used_elem.id =3D le32_to_cpu(used->ring[last_used].id); + used_elem.len =3D le32_to_cpu(used->ring[last_used].len); + + svq->last_used_idx++; + if (unlikely(used_elem.id >=3D svq->vring.num)) { + error_report("Device %s says index %u is used", svq->vdev->name, + used_elem.id); + return NULL; + } + + if (unlikely(!svq->ring_id_maps[used_elem.id])) { + error_report( + "Device %s says index %u is used, but it was not available", + svq->vdev->name, used_elem.id); + return NULL; + } + + descs[used_elem.id].next =3D svq->free_head; + svq->free_head =3D used_elem.id; + + svq->ring_id_maps[used_elem.id]->len =3D used_elem.len; + return g_steal_pointer(&svq->ring_id_maps[used_elem.id]); } =20 -/* Forward vhost notifications */ +static void vhost_svq_flush(VhostShadowVirtqueue *svq, + bool check_for_avail_queue) +{ + VirtQueue *vq =3D svq->vq; + + /* Make as many buffers as possible used. */ + do { + unsigned i =3D 0; + + vhost_svq_set_notification(svq, false); + while (true) { + g_autofree VirtQueueElement *elem =3D vhost_svq_get_buf(svq); + if (!elem) { + break; + } + + if (unlikely(i >=3D svq->vring.num)) { + virtio_error(svq->vdev, + "More than %u used buffers obtained in a %u size = SVQ", + i, svq->vring.num); + virtqueue_fill(vq, elem, elem->len, i); + virtqueue_flush(vq, i); + i =3D 0; + } + virtqueue_fill(vq, elem, elem->len, i++); + } + + virtqueue_flush(vq, i); + event_notifier_set(&svq->svq_call); + + if (check_for_avail_queue && svq->next_guest_avail_elem) { + /* + * Avail ring was full when vhost_svq_flush was called, so it'= s a + * good moment to make more descriptors available if possible + */ + vhost_handle_guest_kick(svq); + } + + vhost_svq_set_notification(svq, true); + } while (vhost_svq_more_used(svq)); +} + +/** + * Forward used buffers. + * + * @n hdev call event notifier, the one that device set to notify svq. + * + * Note that we are not making any buffers available in the loop, there is= no + * way that it runs more than virtqueue size times. + */ static void vhost_svq_handle_call(EventNotifier *n) { VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, hdev_call); =20 - if (unlikely(!event_notifier_test_and_clear(n))) { - return; - } + /* Clear event notifier */ + event_notifier_test_and_clear(n); =20 - event_notifier_set(&svq->svq_call); + vhost_svq_flush(svq, true); } =20 /* @@ -132,6 +468,14 @@ void vhost_svq_set_guest_call_notifier(VhostShadowVirt= queue *svq, int call_fd) event_notifier_init_fd(&svq->svq_call, call_fd); } =20 +/* + * Set the device's memory region notifier. addr =3D NULL clear it. + */ +void vhost_svq_set_host_mr_notifier(VhostShadowVirtqueue *svq, void *addr) +{ + svq->host_notifier_mr =3D addr; +} + /* * Get the shadow vq vring address. * @svq Shadow virtqueue @@ -185,7 +529,8 @@ static void vhost_svq_set_svq_kick_fd_internal(VhostSha= dowVirtqueue *svq, * need to explicitely check for them. */ event_notifier_init_fd(&svq->svq_kick, svq_kick_fd); - event_notifier_set_handler(&svq->svq_kick, vhost_handle_guest_kick); + event_notifier_set_handler(&svq->svq_kick, + vhost_handle_guest_kick_notifier); =20 /* * !check_old means that we are starting SVQ, taking the descriptor fr= om @@ -233,7 +578,16 @@ void vhost_svq_start(struct vhost_dev *dev, unsigned i= dx, void vhost_svq_stop(struct vhost_dev *dev, unsigned idx, VhostShadowVirtqueue *svq) { + unsigned i; event_notifier_set_handler(&svq->svq_kick, NULL); + vhost_svq_flush(svq, false); + + for (i =3D 0; i < svq->vring.num; ++i) { + g_autofree VirtQueueElement *elem =3D svq->ring_id_maps[i]; + if (elem) { + virtqueue_detach_element(svq->vq, elem, elem->len); + } + } } =20 /* @@ -248,7 +602,7 @@ VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *d= ev, int idx) size_t driver_size; size_t device_size; g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); - int r; + int r, i; =20 r =3D event_notifier_init(&svq->hdev_kick, 0); if (r !=3D 0) { @@ -274,6 +628,11 @@ VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *= dev, int idx) memset(svq->vring.desc, 0, driver_size); svq->vring.used =3D qemu_memalign(qemu_real_host_page_size, device_siz= e); memset(svq->vring.used, 0, device_size); + for (i =3D 0; i < num - 1; i++) { + svq->vring.desc[i].next =3D cpu_to_le16(i + 1); + } + + svq->ring_id_maps =3D g_new0(VirtQueueElement *, num); event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); return g_steal_pointer(&svq); =20 @@ -292,6 +651,7 @@ void vhost_svq_free(VhostShadowVirtqueue *vq) event_notifier_cleanup(&vq->hdev_kick); event_notifier_set_handler(&vq->hdev_call, NULL); event_notifier_cleanup(&vq->hdev_call); + g_free(vq->ring_id_maps); qemu_vfree(vq->vring.desc); qemu_vfree(vq->vring.used); g_free(vq); diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index fc8396ba8a..e1c55e43e7 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -19,6 +19,7 @@ #include "hw/virtio/virtio-net.h" #include "hw/virtio/vhost-shadow-virtqueue.h" #include "hw/virtio/vhost-vdpa.h" +#include "hw/virtio/vhost-shadow-virtqueue.h" #include "exec/address-spaces.h" #include "qemu/main-loop.h" #include "cpu.h" @@ -821,6 +822,19 @@ static bool vhost_vdpa_force_iommu(struct vhost_dev *= dev) return true; } =20 +static int vhost_vdpa_vring_pause(struct vhost_dev *dev) +{ + int r; + uint8_t status; + + vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DEVICE_STOPPED); + do { + r =3D vhost_vdpa_call(dev, VHOST_VDPA_GET_STATUS, &status); + } while (r =3D=3D 0 && !(status & VIRTIO_CONFIG_S_DEVICE_STOPPED)); + + return 0; +} + /* * Start or stop a shadow virtqueue in a vdpa device * @@ -844,7 +858,14 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *= dev, unsigned idx, .index =3D vq_index, }; struct vhost_vring_file vhost_call_file =3D { - .index =3D idx + dev->vq_index, + .index =3D vq_index, + }; + struct vhost_vring_addr addr =3D { + .index =3D vq_index, + }; + struct vhost_vring_state num =3D { + .index =3D vq_index, + .num =3D virtio_queue_get_num(dev->vdev, vq_index), }; int r; =20 @@ -852,6 +873,7 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *d= ev, unsigned idx, const EventNotifier *vhost_kick =3D vhost_svq_get_dev_kick_notifie= r(svq); const EventNotifier *vhost_call =3D vhost_svq_get_svq_call_notifie= r(svq); =20 + vhost_svq_get_vring_addr(svq, &addr); if (n->addr) { r =3D virtio_queue_set_host_notifier_mr(dev->vdev, idx, &n->mr, false); @@ -870,8 +892,20 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *= dev, unsigned idx, vhost_kick_file.fd =3D event_notifier_get_fd(vhost_kick); vhost_call_file.fd =3D event_notifier_get_fd(vhost_call); } else { + struct vhost_vring_state state =3D { + .index =3D vq_index, + }; + vhost_svq_stop(dev, idx, svq); =20 + state.num =3D virtio_queue_get_last_avail_idx(dev->vdev, idx); + r =3D vhost_vdpa_set_vring_base(dev, &state); + if (unlikely(r)) { + error_setg_errno(errp, -r, "vhost_set_vring_base failed"); + return false; + } + + vhost_vdpa_vq_get_addr(dev, &addr, &dev->vqs[idx]); if (n->addr) { r =3D virtio_queue_set_host_notifier_mr(dev->vdev, idx, &n->mr, true); @@ -885,6 +919,17 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *= dev, unsigned idx, vhost_call_file.fd =3D v->call_fd[idx]; } =20 + r =3D vhost_vdpa_set_vring_addr(dev, &addr); + if (unlikely(r)) { + error_setg_errno(errp, -r, "vhost_set_vring_addr failed"); + return false; + } + r =3D vhost_vdpa_set_vring_num(dev, &num); + if (unlikely(r)) { + error_setg_errno(errp, -r, "vhost_set_vring_num failed"); + return false; + } + r =3D vhost_vdpa_set_vring_dev_kick(dev, &vhost_kick_file); if (unlikely(r)) { error_setg_errno(errp, -r, "vhost_vdpa_set_vring_kick failed"); @@ -899,6 +944,50 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *= dev, unsigned idx, return true; } =20 +static void vhost_vdpa_get_vq_state(struct vhost_dev *dev, unsigned idx) +{ + struct VirtIODevice *vdev =3D dev->vdev; + + virtio_queue_restore_last_avail_idx(vdev, idx); + virtio_queue_invalidate_signalled_used(vdev, idx); + virtio_queue_update_used_idx(vdev, idx); +} + +/** + * Validate device and guest features against SVQ capabilities + * + * @hdev The vhost device @errp The error + * + * @hdev The hdev + * @svq_features The subset of device features that svq supports. + * @errp The errp + */ +static bool vhost_vdpa_valid_features(struct vhost_dev *hdev, + uint64_t *svq_features, + Error **errp) +{ + uint64_t acked_features =3D hdev->acked_features; + bool ok; + + ok =3D vhost_svq_valid_device_features(svq_features); + if (unlikely(!ok)) { + error_setg(errp, + "Unexpected device feature flags, offered: %"PRIx64", ok: %"PR= Ix64, + hdev->features, *svq_features); + return false; + } + + ok =3D vhost_svq_valid_guest_features(&acked_features); + if (unlikely(!ok)) { + error_setg(errp, + "Invalid guest acked feature flag, acked:%"PRIx64", ok: %"PRIx= 64, + hdev->acked_features, acked_features); + return false; + } + + return true; +} + /** * Enable or disable shadow virtqueue in a vhost vdpa device. * @@ -913,6 +1002,9 @@ void vhost_vdpa_enable_svq(struct vhost_vdpa *v, bool = enable, Error **errp) { struct vhost_dev *hdev =3D v->dev; unsigned n; + int r; + uint64_t svq_features =3D hdev->features | BIT_ULL(VIRTIO_F_IOMMU_PLAT= FORM) | + BIT_ULL(VIRTIO_F_QUEUE_STATE); ERRP_GUARD(); =20 if (enable =3D=3D v->shadow_vqs_enabled) { @@ -920,20 +1012,43 @@ void vhost_vdpa_enable_svq(struct vhost_vdpa *v, boo= l enable, Error **errp) } =20 if (enable) { + bool ok =3D vhost_vdpa_valid_features(hdev, &svq_features, errp); + if (unlikely(!ok)) { + return; + } + /* Allocate resources */ assert(v->shadow_vqs->len =3D=3D 0); for (n =3D 0; n < hdev->nvqs; ++n) { VhostShadowVirtqueue *svq =3D vhost_svq_new(hdev, n); - bool ok; - if (unlikely(!svq)) { error_setg(errp, "Cannot create svq"); enable =3D false; goto err_svq_new; } g_ptr_array_add(v->shadow_vqs, svq); + } + } + + r =3D vhost_vdpa_vring_pause(hdev); + if (unlikely(r)) { + error_setg_errno(errp, -r, "Cannot pause device"); + enable =3D !enable; + goto err_pause; + } + + for (n =3D 0; n < v->shadow_vqs->len; ++n) { + vhost_vdpa_get_vq_state(hdev, hdev->vq_index + n); + } + + /* Reset device so it can be configured */ + vhost_vdpa_dev_start(hdev, false); + + if (enable) { + int r; =20 - ok =3D vhost_vdpa_svq_start_vq(hdev, n, true, errp); + for (n =3D 0; n < v->shadow_vqs->len; ++n) { + bool ok =3D vhost_vdpa_svq_start_vq(hdev, n, true, errp); if (unlikely(!ok)) { /* Free still not started svqs, and go with disable path */ g_ptr_array_set_size(v->shadow_vqs, n); @@ -941,18 +1056,39 @@ void vhost_vdpa_enable_svq(struct vhost_vdpa *v, boo= l enable, Error **errp) break; } } + + /* Need to ack features to set state in vp_vdpa devices */ + r =3D vhost_vdpa_set_features(hdev, svq_features); + if (unlikely(r && !(*errp))) { + error_setg_errno(errp, -r, "Fail to set guest features"); + + /* Go through disable SVQ path */ + enable =3D false; + } } =20 v->shadow_vqs_enabled =3D enable; =20 if (!enable) { + r =3D vhost_vdpa_set_features(hdev, hdev->acked_features | + BIT_ULL(VIRTIO_F_QUEUE_STATE) | + BIT_ULL(VIRTIO_F_IOMMU_PLATFORM)= ); + if (unlikely(r && (!(*errp)))) { + error_setg_errno(errp, -r, "Fail to set guest features"); + } + /* Disable all queues or clean up failed start */ for (n =3D 0; n < v->shadow_vqs->len; ++n) { vhost_vdpa_svq_start_vq(hdev, n, false, *errp ? NULL : errp); } + } =20 + r =3D vhost_vdpa_dev_start(hdev, true); + if (unlikely(r && !(*errp))) { + error_setg_errno(errp, -r, "Fail to start device"); } =20 +err_pause: err_svq_new: if (!enable) { /* Resources cleanup */ --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635534155825291.60253087382955; Fri, 29 Oct 2021 12:02:35 -0700 (PDT) Received: from localhost ([::1]:54212 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgX98-0005gS-NC for importer@patchew.org; Fri, 29 Oct 2021 15:02:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46788) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrX-00067L-EN for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:48552) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrV-000325-HW for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:23 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-589-d0qilp7iNnmGgGfMV0cP0w-1; Fri, 29 Oct 2021 14:44:17 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 510A310A8E04; Fri, 29 Oct 2021 18:44:16 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 30EC65D6CF; Fri, 29 Oct 2021 18:44:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635533061; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5ZojVu0fk8FjansgHvqLHFb4eAzNUMTeqdPgEQIsaxM=; b=KUXdHsmYefZCmRPt+hBkO09TlTFobnmPf8wcrzEKeHJakE+LtMNcDbPBJlvJ8yiDiRQrKW LzrmwY34fBHdnFY51x5TOyeuSlnF7qDYLZoluOAKB6GxH2EM3uE4a/rOVM/WJFlzeEibNo WPjCVYBzwhf0gBDVyz/A9RNGa/eGP58= X-MC-Unique: d0qilp7iNnmGgGfMV0cP0w-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 23/26] util: Add iova_tree_alloc Date: Fri, 29 Oct 2021 20:35:22 +0200 Message-Id: <20211029183525.1776416-24-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635534157032100002 This iova tree function allows it to look for a hole in allocated regions and return a totally new translation for a given translated address. It's usage is mainly to allow devices to access qemu address space, remapping guest's one into a new iova space where qemu can add chunks of addresses. Signed-off-by: Eugenio P=C3=A9rez --- include/qemu/iova-tree.h | 17 +++++ util/iova-tree.c | 139 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 156 insertions(+) diff --git a/include/qemu/iova-tree.h b/include/qemu/iova-tree.h index 8249edd764..33f9b2e13f 100644 --- a/include/qemu/iova-tree.h +++ b/include/qemu/iova-tree.h @@ -29,6 +29,7 @@ #define IOVA_OK (0) #define IOVA_ERR_INVALID (-1) /* Invalid parameters */ #define IOVA_ERR_OVERLAP (-2) /* IOVA range overlapped */ +#define IOVA_ERR_NOMEM (-3) /* Cannot allocate */ =20 typedef struct IOVATree IOVATree; typedef struct DMAMap { @@ -119,6 +120,22 @@ const DMAMap *iova_tree_find_address(const IOVATree *t= ree, hwaddr iova); */ void iova_tree_foreach(IOVATree *tree, iova_tree_iterator iterator); =20 +/** + * iova_tree_alloc: + * + * @tree: the iova tree to allocate from + * @map: the new map (as translated addr & size) to allocate in iova region + * @iova_begin: the minimum address of the allocation + * @iova_end: the maximum addressable direction of the allocation + * + * Allocates a new region of a given size, between iova_min and iova_max. + * + * Return: Same as iova_tree_insert, but cannot overlap and can be out of + * free contiguous range. Caller can get the assigned iova in map->iova. + */ +int iova_tree_alloc(IOVATree *tree, DMAMap *map, hwaddr iova_begin, + hwaddr iova_end); + /** * iova_tree_destroy: * diff --git a/util/iova-tree.c b/util/iova-tree.c index 23ea35b7a4..27c921c4e2 100644 --- a/util/iova-tree.c +++ b/util/iova-tree.c @@ -16,6 +16,36 @@ struct IOVATree { GTree *tree; }; =20 +/* Args to pass to iova_tree_alloc foreach function. */ +struct IOVATreeAllocArgs { + /* Size of the desired allocation */ + size_t new_size; + + /* The minimum address allowed in the allocation */ + hwaddr iova_begin; + + /* The last addressable allowed in the allocation */ + hwaddr iova_last; + + /* Previously-to-last iterated map, can be NULL in the first node */ + const DMAMap *hole_left; + + /* Last iterated map */ + const DMAMap *hole_right; +}; + +/** + * Iterate args to tne next hole + * + * @args The alloc arguments + * @next The next mapping in the tree. Can be NULL to signal the last one + */ +static void iova_tree_alloc_args_iterate(struct IOVATreeAllocArgs *args, + const DMAMap *next) { + args->hole_left =3D args->hole_right; + args->hole_right =3D next; +} + static int iova_tree_compare(gconstpointer a, gconstpointer b, gpointer da= ta) { const DMAMap *m1 =3D a, *m2 =3D b; @@ -107,6 +137,115 @@ int iova_tree_remove(IOVATree *tree, const DMAMap *ma= p) return IOVA_OK; } =20 +/** + * Try to accomodate a map of size ret->size in a hole between + * max(end(hole_left), iova_start). + * + * @args Arguments to allocation + */ +static bool iova_tree_alloc_map_in_hole(const struct IOVATreeAllocArgs *ar= gs) +{ + const DMAMap *left =3D args->hole_left, *right =3D args->hole_right; + uint64_t hole_start, hole_last; + + if (right && right->iova + right->size < args->iova_begin) { + return false; + } + + if (left && left->iova > args->iova_last) { + return false; + } + + hole_start =3D MAX(left ? left->iova + left->size + 1 : 0, args->iova_= begin); + hole_last =3D MIN(right ? right->iova : HWADDR_MAX, args->iova_last); + + if (hole_last - hole_start > args->new_size) { + /* We found a valid hole. */ + return true; + } + + /* Keep iterating */ + return false; +} + +/** + * Foreach dma node in the tree, compare if there is a hole wit its previo= us + * node (or minimum iova address allowed) and the node. + * + * @key Node iterating + * @value Node iterating + * @pargs Struct to communicate with the outside world + * + * Return: false to keep iterating, true if needs break. + */ +static gboolean iova_tree_alloc_traverse(gpointer key, gpointer value, + gpointer pargs) +{ + struct IOVATreeAllocArgs *args =3D pargs; + DMAMap *node =3D value; + + assert(key =3D=3D value); + + iova_tree_alloc_args_iterate(args, node); + if (args->hole_left && args->hole_left->iova > args->iova_last) { + return true; + } + + if (iova_tree_alloc_map_in_hole(args)) { + return true; + } + + return false; +} + +int iova_tree_alloc(IOVATree *tree, DMAMap *map, hwaddr iova_begin, + hwaddr iova_last) +{ + struct IOVATreeAllocArgs args =3D { + .new_size =3D map->size, + .iova_begin =3D iova_begin, + .iova_last =3D iova_last, + }; + + if (iova_begin =3D=3D 0) { + /* Some devices does not like addr 0 */ + iova_begin +=3D qemu_real_host_page_size; + } + + assert(iova_begin < iova_last); + + /* + * Find a valid hole for the mapping + * + * Assuming low iova_begin, so no need to do a binary search to + * locate the first node. + * + * TODO: We can improve the search speed if we save the beginning and = the + * end of holes, so we don't iterate over the previous saved ones. + * + * TODO: Replace all this with g_tree_node_first/next/last when availa= ble + * (from glib since 2.68). To do it with g_tree_foreach complicates the + * code a lot. + * + */ + g_tree_foreach(tree->tree, iova_tree_alloc_traverse, &args); + if (!iova_tree_alloc_map_in_hole(&args)) { + /* + * 2nd try: Last iteration left args->right as the last DMAMap. But + * (right, end) hole needs to be checked too + */ + iova_tree_alloc_args_iterate(&args, NULL); + if (!iova_tree_alloc_map_in_hole(&args)) { + return IOVA_ERR_NOMEM; + } + } + + map->iova =3D MAX(iova_begin, + args.hole_left ? + args.hole_left->iova + args.hole_left->size + 1 : 0); + return iova_tree_insert(tree, map); +} + void iova_tree_destroy(IOVATree *tree) { g_tree_destroy(tree->tree); --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635535024431200.86668843680093; Fri, 29 Oct 2021 12:17:04 -0700 (PDT) Received: from localhost ([::1]:35834 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgXN9-0006QS-3w for importer@patchew.org; Fri, 29 Oct 2021 15:17:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46814) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrc-0006L7-Nm for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:44650) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrZ-000333-Vk for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:28 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-447-RVA87etoMtSukjQ8bYGsdA-1; Fri, 29 Oct 2021 14:44:22 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CD2E3106B7DC; Fri, 29 Oct 2021 18:44:20 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id A0AC65D6CF; Fri, 29 Oct 2021 18:44:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635533065; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MmpsxTCBP+d2avOUHGb/1dqBrc0CAXZvJu7ixmkfDMU=; b=M4TmJvmvb0Uisx6O5dDYI57AO6TuKRIthbrX54OWFXW+q5BCaafvaiqIjoDu03MDrf3HDw bPryOSGw63Q+rjpfMQ5mVwbIDk7NcjP8hEDVRU/ZOYFqGbOLxDdJkJyGS5WW52gE0egvJE 4uyCTrJmno6l9UrjHw7GD7h4uOJEf/g= X-MC-Unique: RVA87etoMtSukjQ8bYGsdA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 24/26] vhost: Add VhostIOVATree Date: Fri, 29 Oct 2021 20:35:23 +0200 Message-Id: <20211029183525.1776416-25-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635535024998100001 This tree is able to look for a translated address from an IOVA address. At first glance is similar to util/iova-tree. However, SVQ working on devices with limited IOVA space need more capabilities, like allocating IOVA chunks or perform reverse translations (qemu addresses to iova). The allocation capability, as "assign a free IOVA address to this chunk of memory in qemu's address space" allows shadow virtqueue to create a new address space that is not restricted by guest's addressable one, so we can allocate shadow vqs vrings outside of its reachability, nor qemu's one. At the moment, the allocation is just done growing, not allowing deletion. It duplicates the tree so it can search efficiently both directions, and it will signal overlap if iova or the translated address is present in it's each array. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-iova-tree.h | 27 +++++++ hw/virtio/vhost-iova-tree.c | 157 ++++++++++++++++++++++++++++++++++++ hw/virtio/meson.build | 2 +- 3 files changed, 185 insertions(+), 1 deletion(-) create mode 100644 hw/virtio/vhost-iova-tree.h create mode 100644 hw/virtio/vhost-iova-tree.c diff --git a/hw/virtio/vhost-iova-tree.h b/hw/virtio/vhost-iova-tree.h new file mode 100644 index 0000000000..56652e7d2b --- /dev/null +++ b/hw/virtio/vhost-iova-tree.h @@ -0,0 +1,27 @@ +/* + * vhost software live migration ring + * + * SPDX-FileCopyrightText: Red Hat, Inc. 2021 + * SPDX-FileContributor: Author: Eugenio P=C3=A9rez + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef HW_VIRTIO_VHOST_IOVA_TREE_H +#define HW_VIRTIO_VHOST_IOVA_TREE_H + +#include "qemu/iova-tree.h" +#include "exec/memory.h" + +typedef struct VhostIOVATree VhostIOVATree; + +VhostIOVATree *vhost_iova_tree_new(uint64_t iova_first, uint64_t iova_last= ); +void vhost_iova_tree_unref(VhostIOVATree *iova_rm); +G_DEFINE_AUTOPTR_CLEANUP_FUNC(VhostIOVATree, vhost_iova_tree_unref); + +const DMAMap *vhost_iova_tree_find_iova(const VhostIOVATree *iova_rm, + const DMAMap *map); +int vhost_iova_tree_map_alloc(VhostIOVATree *iova_rm, DMAMap *map); +void vhost_iova_tree_remove(VhostIOVATree *iova_rm, const DMAMap *map); + +#endif diff --git a/hw/virtio/vhost-iova-tree.c b/hw/virtio/vhost-iova-tree.c new file mode 100644 index 0000000000..021779cfd5 --- /dev/null +++ b/hw/virtio/vhost-iova-tree.c @@ -0,0 +1,157 @@ +/* + * vhost software live migration ring + * + * SPDX-FileCopyrightText: Red Hat, Inc. 2021 + * SPDX-FileContributor: Author: Eugenio P=C3=A9rez + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qemu/iova-tree.h" +#include "vhost-iova-tree.h" + +#define iova_min_addr qemu_real_host_page_size + +/** + * VhostIOVATree, able to: + * - Translate iova address + * - Reverse translate iova address (from translated to iova) + * - Allocate IOVA regions for translated range (potentially slow operatio= n) + * + * Note that it cannot remove nodes. + */ +struct VhostIOVATree { + /* First addresable iova address in the device */ + uint64_t iova_first; + + /* Last addressable iova address in the device */ + uint64_t iova_last; + + /* IOVA address to qemu memory maps. */ + IOVATree *iova_taddr_map; + + /* QEMU virtual memory address to iova maps */ + GTree *taddr_iova_map; +}; + +static gint vhost_iova_tree_cmp_taddr(gconstpointer a, gconstpointer b, + gpointer data) +{ + const DMAMap *m1 =3D a, *m2 =3D b; + + if (m1->translated_addr > m2->translated_addr + m2->size) { + return 1; + } + + if (m1->translated_addr + m1->size < m2->translated_addr) { + return -1; + } + + /* Overlapped */ + return 0; +} + +/** + * Create a new IOVA tree + * + * Returns the new IOVA tree + */ +VhostIOVATree *vhost_iova_tree_new(hwaddr iova_first, hwaddr iova_last) +{ + VhostIOVATree *tree =3D g_new(VhostIOVATree, 1); + + /* Some devices does not like 0 addresses */ + tree->iova_first =3D MAX(iova_first, iova_min_addr); + tree->iova_last =3D iova_last; + + tree->iova_taddr_map =3D iova_tree_new(); + tree->taddr_iova_map =3D g_tree_new_full(vhost_iova_tree_cmp_taddr, NU= LL, + NULL, g_free); + return tree; +} + +/** + * Destroy an IOVA tree + * + * @tree The iova tree + */ +void vhost_iova_tree_unref(VhostIOVATree *tree) +{ + iova_tree_destroy(tree->iova_taddr_map); + g_tree_unref(tree->taddr_iova_map); + g_free(tree); +} + +/** + * Find the IOVA address stored from a memory address + * + * @tree The iova tree + * @map The map with the memory address + * + * Return the stored mapping, or NULL if not found. + */ +const DMAMap *vhost_iova_tree_find_iova(const VhostIOVATree *tree, + const DMAMap *map) +{ + return g_tree_lookup(tree->taddr_iova_map, map); +} + +/** + * Allocate a new mapping + * + * @tree The iova tree + * @map The iova map + * + * Returns: + * - IOVA_OK if the map fits in the container + * - IOVA_ERR_INVALID if the map does not make sense (like size overflow) + * - IOVA_ERR_OVERLAP if the tree already contains that map + * - IOVA_ERR_NOMEM if tree cannot allocate more space. + * + * It returns assignated iova in map->iova if return value is VHOST_DMA_MA= P_OK. + */ +int vhost_iova_tree_map_alloc(VhostIOVATree *tree, DMAMap *map) +{ + DMAMap *new; + int r; + + if (map->translated_addr + map->size < map->translated_addr || + map->perm =3D=3D IOMMU_NONE) { + return IOVA_ERR_INVALID; + } + + /* Check for collisions in translated addresses */ + if (vhost_iova_tree_find_iova(tree, map)) { + return IOVA_ERR_OVERLAP; + } + + /* Allocate a node in IOVA address */ + r =3D iova_tree_alloc(tree->iova_taddr_map, map, tree->iova_first, + tree->iova_last); + if (r !=3D IOVA_OK) { + return r; + } + + /* Allocate node in qemu -> iova translations */ + new =3D g_malloc(sizeof(*new)); + memcpy(new, map, sizeof(*new)); + g_tree_insert(tree->taddr_iova_map, new, new); + return IOVA_OK; +} + +/** + * Remove existing mappings from iova tree + * + * @param iova_rm The vhost iova tree + * @param map The map to remove + */ +void vhost_iova_tree_remove(VhostIOVATree *iova_rm, const DMAMap *map) +{ + const DMAMap *overlap; + + iova_tree_remove(iova_rm->iova_taddr_map, map); + while ((overlap =3D vhost_iova_tree_find_iova(iova_rm, map))) { + g_tree_remove(iova_rm->taddr_iova_map, overlap); + } +} diff --git a/hw/virtio/meson.build b/hw/virtio/meson.build index 2dc87613bc..6047670804 100644 --- a/hw/virtio/meson.build +++ b/hw/virtio/meson.build @@ -11,7 +11,7 @@ softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost-= stub.c')) =20 virtio_ss =3D ss.source_set() virtio_ss.add(files('virtio.c')) -virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backe= nd.c', 'vhost-shadow-virtqueue.c')) +virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backe= nd.c', 'vhost-shadow-virtqueue.c', 'vhost-iova-tree.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER', if_true: files('vhost-user.c')) virtio_ss.add(when: 'CONFIG_VHOST_VDPA', if_true: files('vhost-vdpa.c')) virtio_ss.add(when: 'CONFIG_VIRTIO_BALLOON', if_true: files('virtio-balloo= n.c')) --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635534375755539.108729331173; Fri, 29 Oct 2021 12:06:15 -0700 (PDT) Received: from localhost ([::1]:34632 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgXCg-000339-MQ for importer@patchew.org; Fri, 29 Oct 2021 15:06:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46826) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrg-0006R8-Dg for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:53703) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWre-00033Y-Ln for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:32 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-582-TPrsrzBIMfuL1ZQH41fYbQ-1; Fri, 29 Oct 2021 14:44:26 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4ED9F5721D; Fri, 29 Oct 2021 18:44:25 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id 34B335F4E1; Fri, 29 Oct 2021 18:44:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635533070; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oDdvf2V5s477wTn0WzMmebl1rwoC0/0i5yrK+I/M82E=; b=X1RmX6q3bT9UKiQfABgDKG8ZpGq1kM3W7qwOEsFqEo6675ejivtRn6x9qerFG6ctJ0UTDE Sv5Rrr8vRa2u/5h98Nshn8mbwx1VhG7I/XRqsOqxc9qjt46Q1AYIssFa2UfwJR3vI9RMr5 AkWHl98/OBn7fwsD8UgJN1y0xaLfdvs= X-MC-Unique: TPrsrzBIMfuL1ZQH41fYbQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 25/26] vhost: Use a tree to store memory mappings Date: Fri, 29 Oct 2021 20:35:24 +0200 Message-Id: <20211029183525.1776416-26-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635534376542100001 Track memory translations of devices with IOMMU (all vhost-vdpa devices at the moment). It does not work if device has restrictions in its iova range at the moment. Updates to tree are protected by BQL, each one always run from main event loop context. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-vdpa.h | 3 ++ hw/virtio/vhost-vdpa.c | 50 +++++++++++++++++++++++++++++++++- 2 files changed, 52 insertions(+), 1 deletion(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 2f57b17208..365b102c14 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -14,6 +14,7 @@ =20 #include =20 +#include "hw/virtio/vhost-iova-tree.h" #include "hw/virtio/virtio.h" #include "standard-headers/linux/vhost_types.h" =20 @@ -30,6 +31,8 @@ typedef struct vhost_vdpa { MemoryListener listener; struct vhost_vdpa_iova_range iova_range; bool shadow_vqs_enabled; + /* IOVA mapping used by Shadow Virtqueue */ + VhostIOVATree *iova_map; GPtrArray *shadow_vqs; struct vhost_dev *dev; int kick_fd[VIRTIO_QUEUE_MAX]; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index e1c55e43e7..a827ecbe4f 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -466,6 +466,7 @@ static void vhost_vdpa_svq_cleanup(struct vhost_dev *de= v) vhost_svq_stop(dev, idx, g_ptr_array_index(v->shadow_vqs, idx)); } g_ptr_array_free(v->shadow_vqs, true); + g_clear_pointer(&v->iova_map, vhost_iova_tree_unref); } =20 static int vhost_vdpa_cleanup(struct vhost_dev *dev) @@ -822,6 +823,22 @@ static bool vhost_vdpa_force_iommu(struct vhost_dev *= dev) return true; } =20 +/** + * Maps QEMU vaddr memory to device in a suitable way for shadow virtqueue: + * - It always reference qemu memory address, not guest's memory. + * - TODO It's always in range of device. + * + * It returns the translated address + */ +static int vhost_vdpa_svq_map(struct vhost_vdpa *v, DMAMap *map) +{ + int r =3D vhost_iova_tree_map_alloc(v->iova_map, map); + assert(r =3D=3D IOVA_OK); + + return vhost_vdpa_dma_map(v, map->iova, map->size, + (void *)map->translated_addr, false); +} + static int vhost_vdpa_vring_pause(struct vhost_dev *dev) { int r; @@ -872,8 +889,36 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *= dev, unsigned idx, if (svq_mode) { const EventNotifier *vhost_kick =3D vhost_svq_get_dev_kick_notifie= r(svq); const EventNotifier *vhost_call =3D vhost_svq_get_svq_call_notifie= r(svq); + DMAMap device_region, driver_region; =20 vhost_svq_get_vring_addr(svq, &addr); + driver_region =3D (DMAMap) { + .translated_addr =3D (hwaddr)addr.desc_user_addr, + + /* + * DMAMAp.size include the last byte included in the range, wh= ile + * sizeof marks one past it. Substract one byte to make them m= atch. + */ + .size =3D vhost_svq_driver_area_size(svq) - 1, + .perm =3D VHOST_ACCESS_RO, + }; + device_region =3D (DMAMap) { + .translated_addr =3D (hwaddr)addr.used_user_addr, + .size =3D vhost_svq_device_area_size(svq) - 1, + .perm =3D VHOST_ACCESS_RW, + }; + + r =3D vhost_vdpa_svq_map(v, &driver_region); + assert(r =3D=3D 0); + r =3D vhost_vdpa_svq_map(v, &device_region); + assert(r =3D=3D 0); + + /* Expose IOVA addresses to vDPA device */ + addr.avail_user_addr =3D driver_region.iova + addr.avail_user_addr + - addr.desc_user_addr; + addr.desc_user_addr =3D driver_region.iova; + addr.used_user_addr =3D device_region.iova; + if (n->addr) { r =3D virtio_queue_set_host_notifier_mr(dev->vdev, idx, &n->mr, false); @@ -885,7 +930,6 @@ static bool vhost_vdpa_svq_start_vq(struct vhost_dev *d= ev, unsigned idx, assert(r =3D=3D 0); vhost_svq_set_host_mr_notifier(svq, n->addr); } - vhost_svq_set_guest_call_notifier(svq, v->call_fd[idx]); vhost_svq_start(dev, idx, svq, v->kick_fd[idx]); =20 @@ -1001,6 +1045,7 @@ static bool vhost_vdpa_valid_features(struct vhost_de= v *hdev, void vhost_vdpa_enable_svq(struct vhost_vdpa *v, bool enable, Error **errp) { struct vhost_dev *hdev =3D v->dev; + hwaddr iova_first =3D v->iova_range.first, iova_last =3D v->iova_range= .last; unsigned n; int r; uint64_t svq_features =3D hdev->features | BIT_ULL(VIRTIO_F_IOMMU_PLAT= FORM) | @@ -1017,6 +1062,8 @@ void vhost_vdpa_enable_svq(struct vhost_vdpa *v, bool= enable, Error **errp) return; } =20 + v->iova_map =3D vhost_iova_tree_new(iova_first, iova_last); + /* Allocate resources */ assert(v->shadow_vqs->len =3D=3D 0); for (n =3D 0; n < hdev->nvqs; ++n) { @@ -1093,6 +1140,7 @@ err_svq_new: if (!enable) { /* Resources cleanup */ g_ptr_array_set_size(v->shadow_vqs, 0); + g_clear_pointer(&v->iova_map, vhost_iova_tree_unref); } } =20 --=20 2.27.0 From nobody Sun Feb 8 20:53:06 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1635534527411558.8392408906268; Fri, 29 Oct 2021 12:08:47 -0700 (PDT) Received: from localhost ([::1]:43676 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mgXF8-0000oJ-2O for importer@patchew.org; Fri, 29 Oct 2021 15:08:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46838) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrm-0006bQ-5b for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:47441) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mgWrj-00034o-Vs for qemu-devel@nongnu.org; Fri, 29 Oct 2021 14:44:37 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-588-lH-zJimNPq-DqGmDYcr4Mw-1; Fri, 29 Oct 2021 14:44:31 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B1E6618414A0; Fri, 29 Oct 2021 18:44:29 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.75]) by smtp.corp.redhat.com (Postfix) with ESMTP id A2A475D6CF; Fri, 29 Oct 2021 18:44:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635533074; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1LVxG3kH5/l5zYFbZvNOuy5iveaPCDv0YsOE7we2KI4=; b=XXqtC9ZNA9OFfgOxS2lTY2gIFvVjD+J4QcPtGwO+FPtxK4Al4mh+6hAtGyqs0wfeCA3sgN 8jXo9FtrpnUUHB3r1Bl8V3e2ZeNfOxDIQbzoqtXAUVnF8kbu8ycsRBANNixz8x/Xwutq97 K8ume/aNaqVeN6gGxv0JUeDKMHas9ik= X-MC-Unique: lH-zJimNPq-DqGmDYcr4Mw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC PATCH v5 26/26] vdpa: Add custom IOTLB translations to SVQ Date: Fri, 29 Oct 2021 20:35:25 +0200 Message-Id: <20211029183525.1776416-27-eperezma@redhat.com> In-Reply-To: <20211029183525.1776416-1-eperezma@redhat.com> References: <20211029183525.1776416-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Richard Henderson , Stefan Hajnoczi , Peter Xu , Markus Armbruster , Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Paolo Bonzini , Stefano Garzarella , Eric Blake , virtualization@lists.linux-foundation.org, Eduardo Habkost Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) X-ZM-MESSAGEID: 1635534528234100001 Use translations added in VhostIOVATree in SVQ. Now every element needs to remember the iova / GPA address also, so VirtQueue can consume the elements properly. This adds a little overhead per VQ element, having to allocate more memory to stash them. As a possible optimization, this allocation could be avoided if the descriptor is not a chain but a single one, but this is left undone. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 4 +- hw/virtio/vhost-shadow-virtqueue.c | 156 ++++++++++++++++++++++------- hw/virtio/vhost-vdpa.c | 35 ++++++- 3 files changed, 159 insertions(+), 36 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index ac55588009..903b9f7a14 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -12,6 +12,7 @@ =20 #include "hw/virtio/vhost.h" #include "qemu/event_notifier.h" +#include "hw/virtio/vhost-iova-tree.h" =20 typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 @@ -35,7 +36,8 @@ void vhost_svq_start(struct vhost_dev *dev, unsigned idx, void vhost_svq_stop(struct vhost_dev *dev, unsigned idx, VhostShadowVirtqueue *svq); =20 -VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx); +VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx, + VhostIOVATree *iova_map); =20 void vhost_svq_free(VhostShadowVirtqueue *vq); =20 diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index ad1b2342be..7ab506f9e7 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -11,12 +11,19 @@ #include "hw/virtio/vhost-shadow-virtqueue.h" #include "hw/virtio/vhost.h" #include "hw/virtio/virtio-access.h" +#include "hw/virtio/vhost-iova-tree.h" =20 #include "standard-headers/linux/vhost_types.h" =20 #include "qemu/error-report.h" #include "qemu/main-loop.h" =20 +typedef struct SVQElement { + VirtQueueElement elem; + void **vaddr_in_sg; + void **vaddr_out_sg; +} SVQElement; + /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { /* Shadow vring */ @@ -49,11 +56,14 @@ typedef struct VhostShadowVirtqueue { /* Virtio device */ VirtIODevice *vdev; =20 + /* IOVA mapping if used */ + VhostIOVATree *iova_map; + /* Map for returning guest's descriptors */ - VirtQueueElement **ring_id_maps; + SVQElement **ring_id_maps; =20 - /* Next VirtQueue element that guest made available */ - VirtQueueElement *next_guest_avail_elem; + /* Next SVQ element that guest made available */ + SVQElement *next_guest_avail_elem; =20 /* Next head to expose to device */ uint16_t avail_idx_shadow; @@ -80,6 +90,14 @@ const EventNotifier *vhost_svq_get_dev_kick_notifier( return &svq->hdev_kick; } =20 +static void vhost_svq_elem_free(SVQElement *elem) +{ + g_free(elem->vaddr_in_sg); + g_free(elem->vaddr_out_sg); + g_free(elem); +} +G_DEFINE_AUTOPTR_CLEANUP_FUNC(SVQElement, vhost_svq_elem_free) + /** * VirtIO transport device feature acknowledge * @@ -99,13 +117,7 @@ bool vhost_svq_valid_device_features(uint64_t *dev_feat= ures) continue; =20 case VIRTIO_F_ACCESS_PLATFORM: - /* SVQ does not know how to translate addresses */ - if (*dev_features & BIT_ULL(b)) { - clear_bit(b, dev_features); - r =3D false; - } - break; - + /* SVQ trust in host's IOMMU to translate addresses */ case VIRTIO_F_VERSION_1: /* SVQ trust that guest vring is little endian */ if (!(*dev_features & BIT_ULL(b))) { @@ -175,7 +187,49 @@ static void vhost_svq_set_notification(VhostShadowVirt= queue *svq, bool enable) } } =20 +static bool vhost_svq_translate_addr(const VhostShadowVirtqueue *svq, + void ***vaddr, const struct iovec *io= vec, + size_t num) +{ + size_t i; + + if (num =3D=3D 0) { + return true; + } + + g_autofree void **addrs =3D g_new(void *, num); + for (i =3D 0; i < num; ++i) { + DMAMap needle =3D { + .translated_addr =3D (hwaddr)iovec[i].iov_base, + .size =3D iovec[i].iov_len, + }; + size_t off; + + const DMAMap *map =3D vhost_iova_tree_find_iova(svq->iova_map, &ne= edle); + /* + * Map cannot be NULL since iova map contains all guest space and + * qemu already has a physical address mapped + */ + if (unlikely(!map)) { + error_report("Invalid address 0x%"HWADDR_PRIx" given by guest", + needle.translated_addr); + return false; + } + + /* + * Map->iova chunk size is ignored. What to do if descriptor + * (addr, size) does not fit is delegated to the device. + */ + off =3D needle.translated_addr - map->translated_addr; + addrs[i] =3D (void *)(map->iova + off); + } + + *vaddr =3D g_steal_pointer(&addrs); + return true; +} + static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, + void * const *vaddr_sg, const struct iovec *iovec, size_t num, bool more_descs, bool writ= e) { @@ -194,7 +248,7 @@ static void vhost_vring_write_descs(VhostShadowVirtqueu= e *svq, } else { descs[i].flags =3D flags; } - descs[i].addr =3D cpu_to_le64((hwaddr)iovec[n].iov_base); + descs[i].addr =3D cpu_to_le64((hwaddr)vaddr_sg[n]); descs[i].len =3D cpu_to_le32(iovec[n].iov_len); =20 last =3D i; @@ -204,43 +258,62 @@ static void vhost_vring_write_descs(VhostShadowVirtqu= eue *svq, svq->free_head =3D le16_to_cpu(descs[last].next); } =20 -static unsigned vhost_svq_add_split(VhostShadowVirtqueue *svq, - VirtQueueElement *elem) +static bool vhost_svq_add_split(VhostShadowVirtqueue *svq, + SVQElement *svq_elem, + unsigned *head) { - int head; + VirtQueueElement *elem =3D &svq_elem->elem; unsigned avail_idx; vring_avail_t *avail =3D svq->vring.avail; + bool ok; =20 - head =3D svq->free_head; + *head =3D svq->free_head; =20 /* We need some descriptors here */ assert(elem->out_num || elem->in_num); =20 - vhost_vring_write_descs(svq, elem->out_sg, elem->out_num, - elem->in_num > 0, false); - vhost_vring_write_descs(svq, elem->in_sg, elem->in_num, false, true); + ok =3D vhost_svq_translate_addr(svq, &svq_elem->vaddr_in_sg, elem->in_= sg, + elem->in_num); + if (unlikely(!ok)) { + return false; + } + ok =3D vhost_svq_translate_addr(svq, &svq_elem->vaddr_out_sg, elem->ou= t_sg, + elem->out_num); + if (unlikely(!ok)) { + return false; + } + + vhost_vring_write_descs(svq, svq_elem->vaddr_out_sg, elem->out_sg, + elem->out_num, elem->in_num > 0, false); + vhost_vring_write_descs(svq, svq_elem->vaddr_in_sg, elem->in_sg, + elem->in_num, false, true); =20 /* * Put entry in available array (but don't update avail->idx until they * do sync). */ avail_idx =3D svq->avail_idx_shadow & (svq->vring.num - 1); - avail->ring[avail_idx] =3D cpu_to_le16(head); + avail->ring[avail_idx] =3D cpu_to_le16(*head); svq->avail_idx_shadow++; =20 /* Update avail index after the descriptor is wrote */ smp_wmb(); avail->idx =3D cpu_to_le16(svq->avail_idx_shadow); =20 - return head; + return true; =20 } =20 -static void vhost_svq_add(VhostShadowVirtqueue *svq, VirtQueueElement *ele= m) +static bool vhost_svq_add(VhostShadowVirtqueue *svq, SVQElement *elem) { - unsigned qemu_head =3D vhost_svq_add_split(svq, elem); + unsigned qemu_head; + bool ok =3D vhost_svq_add_split(svq, elem, &qemu_head); + if (unlikely(!ok)) { + return false; + } =20 svq->ring_id_maps[qemu_head] =3D elem; + return true; } =20 static void vhost_svq_kick(VhostShadowVirtqueue *svq) @@ -284,7 +357,8 @@ static void vhost_handle_guest_kick(VhostShadowVirtqueu= e *svq) } =20 while (true) { - VirtQueueElement *elem; + SVQElement *elem; + bool ok; =20 if (svq->next_guest_avail_elem) { elem =3D g_steal_pointer(&svq->next_guest_avail_elem); @@ -296,7 +370,7 @@ static void vhost_handle_guest_kick(VhostShadowVirtqueu= e *svq) break; } =20 - if (elem->out_num + elem->in_num > + if (elem->elem.out_num + elem->elem.in_num > vhost_svq_available_slots(svq)) { /* * This condition is possible since a contiguous buffer in= GPA @@ -313,7 +387,11 @@ static void vhost_handle_guest_kick(VhostShadowVirtque= ue *svq) return; } =20 - vhost_svq_add(svq, elem); + ok =3D vhost_svq_add(svq, elem); + if (unlikely(!ok)) { + /* VQ is broken, just return and ignore any other kicks */ + return; + } vhost_svq_kick(svq); } =20 @@ -344,7 +422,7 @@ static bool vhost_svq_more_used(VhostShadowVirtqueue *s= vq) return svq->last_used_idx !=3D svq->shadow_used_idx; } =20 -static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq) +static SVQElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq) { vring_desc_t *descs =3D svq->vring.desc; const vring_used_t *used =3D svq->vring.used; @@ -378,7 +456,7 @@ static VirtQueueElement *vhost_svq_get_buf(VhostShadowV= irtqueue *svq) descs[used_elem.id].next =3D svq->free_head; svq->free_head =3D used_elem.id; =20 - svq->ring_id_maps[used_elem.id]->len =3D used_elem.len; + svq->ring_id_maps[used_elem.id]->elem.len =3D used_elem.len; return g_steal_pointer(&svq->ring_id_maps[used_elem.id]); } =20 @@ -393,8 +471,9 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, =20 vhost_svq_set_notification(svq, false); while (true) { - g_autofree VirtQueueElement *elem =3D vhost_svq_get_buf(svq); - if (!elem) { + g_autofree SVQElement *svq_elem =3D vhost_svq_get_buf(svq); + VirtQueueElement *elem; + if (!svq_elem) { break; } =20 @@ -406,6 +485,7 @@ static void vhost_svq_flush(VhostShadowVirtqueue *svq, virtqueue_flush(vq, i); i =3D 0; } + elem =3D &svq_elem->elem; virtqueue_fill(vq, elem, elem->len, i++); } =20 @@ -583,10 +663,15 @@ void vhost_svq_stop(struct vhost_dev *dev, unsigned i= dx, vhost_svq_flush(svq, false); =20 for (i =3D 0; i < svq->vring.num; ++i) { - g_autofree VirtQueueElement *elem =3D svq->ring_id_maps[i]; - if (elem) { - virtqueue_detach_element(svq->vq, elem, elem->len); + g_autofree SVQElement *svq_elem =3D svq->ring_id_maps[i]; + VirtQueueElement *elem; + + if (!svq_elem) { + continue; } + + elem =3D &svq_elem->elem; + virtqueue_detach_element(svq->vq, elem, elem->len); } } =20 @@ -594,7 +679,8 @@ void vhost_svq_stop(struct vhost_dev *dev, unsigned idx, * Creates vhost shadow virtqueue, and instruct vhost device to use the sh= adow * methods and file descriptors. */ -VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx) +VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev *dev, int idx, + VhostIOVATree *iova_map) { int vq_idx =3D dev->vq_index + idx; unsigned num =3D virtio_queue_get_num(dev->vdev, vq_idx); @@ -628,11 +714,13 @@ VhostShadowVirtqueue *vhost_svq_new(struct vhost_dev = *dev, int idx) memset(svq->vring.desc, 0, driver_size); svq->vring.used =3D qemu_memalign(qemu_real_host_page_size, device_siz= e); memset(svq->vring.used, 0, device_size); + svq->iova_map =3D iova_map; + for (i =3D 0; i < num - 1; i++) { svq->vring.desc[i].next =3D cpu_to_le16(i + 1); } =20 - svq->ring_id_maps =3D g_new0(VirtQueueElement *, num); + svq->ring_id_maps =3D g_new0(SVQElement *, num); event_notifier_set_handler(&svq->hdev_call, vhost_svq_handle_call); return g_steal_pointer(&svq); =20 diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index a827ecbe4f..8466580ae7 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -210,6 +210,18 @@ static void vhost_vdpa_listener_region_add(MemoryListe= ner *listener, vaddr, section->readonly); =20 llsize =3D int128_sub(llend, int128_make64(iova)); + if (v->shadow_vqs_enabled) { + DMAMap mem_region =3D { + .translated_addr =3D (hwaddr)vaddr, + .size =3D int128_get64(llsize) - 1, + .perm =3D IOMMU_ACCESS_FLAG(true, section->readonly), + }; + + int r =3D vhost_iova_tree_map_alloc(v->iova_map, &mem_region); + assert(r =3D=3D IOVA_OK); + + iova =3D mem_region.iova; + } =20 vhost_vdpa_iotlb_batch_begin_once(v); ret =3D vhost_vdpa_dma_map(v, iova, int128_get64(llsize), @@ -262,6 +274,20 @@ static void vhost_vdpa_listener_region_del(MemoryListe= ner *listener, =20 llsize =3D int128_sub(llend, int128_make64(iova)); =20 + if (v->shadow_vqs_enabled) { + const DMAMap *result; + const void *vaddr =3D memory_region_get_ram_ptr(section->mr) + + section->offset_within_region + + (iova - section->offset_within_address_space); + DMAMap mem_region =3D { + .translated_addr =3D (hwaddr)vaddr, + .size =3D int128_get64(llsize) - 1, + }; + + result =3D vhost_iova_tree_find_iova(v->iova_map, &mem_region); + iova =3D result->iova; + vhost_iova_tree_remove(v->iova_map, &mem_region); + } vhost_vdpa_iotlb_batch_begin_once(v); ret =3D vhost_vdpa_dma_unmap(v, iova, int128_get64(llsize)); if (ret) { @@ -1067,7 +1093,7 @@ void vhost_vdpa_enable_svq(struct vhost_vdpa *v, bool= enable, Error **errp) /* Allocate resources */ assert(v->shadow_vqs->len =3D=3D 0); for (n =3D 0; n < hdev->nvqs; ++n) { - VhostShadowVirtqueue *svq =3D vhost_svq_new(hdev, n); + VhostShadowVirtqueue *svq =3D vhost_svq_new(hdev, n, v->iova_m= ap); if (unlikely(!svq)) { error_setg(errp, "Cannot create svq"); enable =3D false; @@ -1088,6 +1114,13 @@ void vhost_vdpa_enable_svq(struct vhost_vdpa *v, boo= l enable, Error **errp) vhost_vdpa_get_vq_state(hdev, hdev->vq_index + n); } =20 + memory_listener_unregister(&v->listener); + r =3D vhost_vdpa_dma_unmap(v, iova_first, + (iova_last - iova_first) & TARGET_PAGE_MASK); + if (unlikely(r)) { + error_setg_errno(errp, -r, "Fail to invalidate IOTLB"); + } + /* Reset device so it can be configured */ vhost_vdpa_dev_start(hdev, false); =20 --=20 2.27.0