From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621441899; cv=none; d=zohomail.com; s=zohoarc; b=Hws1iXTcdihywuOYGwtn6hB0iz9GcaTX7ITCgyHECwEc8+6LP3G41cLVRPgCHKIlXQCGqYxpnJgBfGc9weJEejyOPjcSMjoc2OMZCZCCTksAkuSsI5C57/XjJz+BgSab4//dNd5WVffR56yeiMoxorsTZB19t0sWr2Brpj/2uTU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621441899; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0kky/RYc9A8VMyn0hQWQkkpfFNLpVtNdxBiYEiHpkaQ=; b=nPtVyAxuXBhI70Z22+sH/gS7H/9gz+RwmoNZWguTSYeK4zEUlUjuOKyXTdgraq7IfEvAMVIpERkHt47YIZ80ARkzM1IFZfXnCyLiRB5SKVarW1mEFxR8cxOYdDfyshWXB4gD193l7FOygcRF6rNyctfLLIgn3TLOYikrlsc1IZU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 162144189931778.84857065150095; Wed, 19 May 2021 09:31:39 -0700 (PDT) Received: from localhost ([::1]:56188 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljP6e-0004WF-W1 for importer@patchew.org; Wed, 19 May 2021 12:31:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34266) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP4i-00021k-Ir for qemu-devel@nongnu.org; Wed, 19 May 2021 12:29:36 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:57427) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP4f-00054g-DP for qemu-devel@nongnu.org; Wed, 19 May 2021 12:29:35 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-62-lhhaU_8DPOCyP62SY4dwTw-1; Wed, 19 May 2021 12:29:29 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D39B1180FD6A; Wed, 19 May 2021 16:29:27 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 849565D6AC; Wed, 19 May 2021 16:29:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441772; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0kky/RYc9A8VMyn0hQWQkkpfFNLpVtNdxBiYEiHpkaQ=; b=eJDh0zhZOXD98hUWOoWntl8vsO4jJJ7Nqfxw6kWvPNRYUSIGCZc4muUichlRGzvw0RXCI6 4glhJznqsKrY+jcFGfGk0+s+qCaO9mymajS13LyTdtw1p94SMqGLBEoGiMGWRhUHfapUls 6AgUdm0VyRrwq5FKQRxVBkI2fxodhds= X-MC-Unique: lhhaU_8DPOCyP62SY4dwTw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 01/29] virtio: Add virtio_queue_is_host_notifier_enabled Date: Wed, 19 May 2021 18:28:35 +0200 Message-Id: <20210519162903.1172366-2-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This allows shadow virtqueue code to assert the queue status before making changes. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/virtio.h | 1 + hw/virtio/virtio.c | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index b7ece7a6a8..c2c7cee993 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -316,6 +316,7 @@ void virtio_device_release_ioeventfd(VirtIODevice *vdev= ); bool virtio_device_ioeventfd_enabled(VirtIODevice *vdev); EventNotifier *virtio_queue_get_host_notifier(VirtQueue *vq); void virtio_queue_set_host_notifier_enabled(VirtQueue *vq, bool enabled); +bool virtio_queue_is_host_notifier_enabled(const VirtQueue *vq); void virtio_queue_host_notifier_read(EventNotifier *n); void virtio_queue_aio_set_host_notifier_handler(VirtQueue *vq, AioContext = *ctx, VirtIOHandleAIOOutput hand= le_output); diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 07f4e60b30..a86b3f9c26 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -3594,6 +3594,11 @@ EventNotifier *virtio_queue_get_host_notifier(VirtQu= eue *vq) return &vq->host_notifier; } =20 +bool virtio_queue_is_host_notifier_enabled(const VirtQueue *vq) +{ + return vq->host_notifier_enabled; +} + void virtio_queue_set_host_notifier_enabled(VirtQueue *vq, bool enabled) { vq->host_notifier_enabled =3D enabled; --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621441929; cv=none; d=zohomail.com; s=zohoarc; b=nwuexBtvrkI4sv4JdXHvfjCF3PUcEWnhQeiMf04oHjghVwiDo6TJEd2eQIM4Y51rSiVN149/A9aQbvJKoL/MWmvknutMztO15KmCt1+ZNb61rNHbmNXTmxmDmWMJUlRaoGeK+C11QyGWSv3gQP1bWNLX4Xfsih7qX1SacId1IkE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621441929; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=+kHOjQsXC/K2n38pcbsFNStVeupnDFmkXVqAbKuxVWo=; b=cC9aO7RJ1m/z7Qk+dHovqcPmWotavI1Dxq0XelFtLn1xdcAo2SIvrBHEM4H9rij19h2KWeEV4Gh28pzSO2ZJU8YwGOUukGkHZDhxHanu39VUZkT3ujclGGkLEWN6zndKC0W+fxFEHN6h/VxdKQLU+B9RQjF0MSID4T8ioLaBkwM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621441929582984.7430586518617; Wed, 19 May 2021 09:32:09 -0700 (PDT) Received: from localhost ([::1]:56858 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljP7A-0004zC-H1 for importer@patchew.org; Wed, 19 May 2021 12:32:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34296) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP4n-0002Al-Ig for qemu-devel@nongnu.org; Wed, 19 May 2021 12:29:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:50966) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP4m-00057l-3s for qemu-devel@nongnu.org; Wed, 19 May 2021 12:29:41 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-50-HfP3HZ3COZWfBf3K6Djmdw-1; Wed, 19 May 2021 12:29:36 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C9F7C1005E40; Wed, 19 May 2021 16:29:34 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4C1C15D6AC; Wed, 19 May 2021 16:29:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441779; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+kHOjQsXC/K2n38pcbsFNStVeupnDFmkXVqAbKuxVWo=; b=dAjnADT1bCapyKAUEK6dJkpyb/Gp+8GYH4ojRBuafc9jhampQYLzPAmOMXGo0/vzifVLE3 8mWYEdXzjD1zzWoP6NS6Q/7asTP/icdeIo1wEelE4aqflB0SRvz7pe9Z9T5t8L+d9o3xs1 07gfgGFwcSS0QAoYPQaONBOKGNJZn7A= X-MC-Unique: HfP3HZ3COZWfBf3K6Djmdw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 02/29] vhost: Save masked_notifier state Date: Wed, 19 May 2021 18:28:36 +0200 Message-Id: <20210519162903.1172366-3-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) It will be used to configure shadow virtqueue. Shadow virtqueue will relay the device->guest notifications, so vhost need to be able to tell the masking status. Reviewed-by: Stefan Hajnoczi Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost.h | 1 + hw/virtio/vhost.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 4a8bc75415..ac963bf23d 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -28,6 +28,7 @@ struct vhost_virtqueue { unsigned avail_size; unsigned long long used_phys; unsigned used_size; + bool notifier_is_masked; EventNotifier masked_notifier; struct vhost_dev *dev; }; diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index bbc2f228b5..40f9f64ebd 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1527,6 +1527,8 @@ void vhost_virtqueue_mask(struct vhost_dev *hdev, Vir= tIODevice *vdev, int n, r =3D hdev->vhost_ops->vhost_set_vring_call(hdev, &file); if (r < 0) { VHOST_OPS_DEBUG("vhost_set_vring_call failed"); + } else { + hdev->vqs[index].notifier_is_masked =3D mask; } } =20 --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621441934; cv=none; d=zohomail.com; s=zohoarc; b=NV4nZ+rMVd0aAAdA1wT9VlfL15F/T73TWyE7WUkRGSzNz6qGc+Z9T5UPaT3WEytp2/0vEWabqp7UNxCrE017o3UEfKNEClUn/FYvSJbQnOP3ffCDb71qSJg4Ft1/b7C0PENP9mDAKPQrl3BReNIe4sNaeQAqo7MVBPf92PJs4WU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621441934; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=yJUB2BKxUp+/hgtNUy8u9evkox43m4+lW00bD5Y36Zs=; b=gT5lg7AWdn2DAu45lE4yW0/jG4tDbdKlpiNwq50uAymmjb2+d0DmXgivrs41neXNI8C0fCBi8RUJFLohqi0lUyF0uiHjBRLFzSkhJYBFo6D0Um0B5tsFYuoqk/Dft86eT7NpREw4HviUG0aysxVetjSy9lSparn0wo1UmM1QvXE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621441934252461.6417495822959; Wed, 19 May 2021 09:32:14 -0700 (PDT) Received: from localhost ([::1]:57204 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljP7F-0005E3-5e for importer@patchew.org; Wed, 19 May 2021 12:32:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34332) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP4u-0002Pm-D0 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:29:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:53326) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP4s-0005A7-4y for qemu-devel@nongnu.org; Wed, 19 May 2021 12:29:48 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-471-YhApobXUMKS7NlYxLmHjkg-1; Wed, 19 May 2021 12:29:43 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1B0F88015C6; Wed, 19 May 2021 16:29:42 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3B4535D6D5; Wed, 19 May 2021 16:29:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441785; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yJUB2BKxUp+/hgtNUy8u9evkox43m4+lW00bD5Y36Zs=; b=aXhboMB2qnppHRK8zHlGyOQJGCnUevI+OJYtOwCRdwOFQrdtSOfAwvkR+YVCdpYZZiJ7KO HDbQjvs548RtXVNSV6Sq275YoiON2dme2rcbgPGAK+JNh1mZiOtDqovYRitRVpyhpbLfkK bQxjtorj8oytGK8y9CaTpsFwAr3UyOY= X-MC-Unique: YhApobXUMKS7NlYxLmHjkg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 03/29] vhost: Add VhostShadowVirtqueue Date: Wed, 19 May 2021 18:28:37 +0200 Message-Id: <20210519162903.1172366-4-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Vhost shadow virtqueue (SVQ)is an intermediate jump for virtqueue notifications and buffers, allowing qemu to track them. While qemu is forwarding the buffers and virtqueue changes, is able to commit the memory it's being dirtied, the same way regular qemu's VirtIO devices do. This commit only exposes basic SVQ allocation and free, so changes regarding different aspects of SVQ (notifications forwarding, buffer forwarding, starting/stopping) are more isolated and easier to bisect. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 24 ++++++++++++ hw/virtio/vhost-shadow-virtqueue.c | 63 ++++++++++++++++++++++++++++++ hw/virtio/meson.build | 2 +- 3 files changed, 88 insertions(+), 1 deletion(-) create mode 100644 hw/virtio/vhost-shadow-virtqueue.h create mode 100644 hw/virtio/vhost-shadow-virtqueue.c diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h new file mode 100644 index 0000000000..6cc18d6acb --- /dev/null +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -0,0 +1,24 @@ +/* + * vhost software live migration ring + * + * SPDX-FileCopyrightText: Red Hat, Inc. 2021 + * SPDX-FileContributor: Author: Eugenio P=C3=A9rez + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef VHOST_SHADOW_VIRTQUEUE_H +#define VHOST_SHADOW_VIRTQUEUE_H + +#include "qemu/osdep.h" + +#include "hw/virtio/virtio.h" +#include "hw/virtio/vhost.h" + +typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; + +VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_dev *dev, int idx); + +void vhost_shadow_vq_free(VhostShadowVirtqueue *vq); + +#endif diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c new file mode 100644 index 0000000000..4512e5b058 --- /dev/null +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -0,0 +1,63 @@ +/* + * vhost software live migration ring + * + * SPDX-FileCopyrightText: Red Hat, Inc. 2021 + * SPDX-FileContributor: Author: Eugenio P=C3=A9rez + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "hw/virtio/vhost-shadow-virtqueue.h" + +#include "qemu/error-report.h" +#include "qemu/event_notifier.h" + +/* Shadow virtqueue to relay notifications */ +typedef struct VhostShadowVirtqueue { + /* Shadow kick notifier, sent to vhost */ + EventNotifier kick_notifier; + /* Shadow call notifier, sent to vhost */ + EventNotifier call_notifier; +} VhostShadowVirtqueue; + +/* + * Creates vhost shadow virtqueue, and instruct vhost device to use the sh= adow + * methods and file descriptors. + */ +VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_dev *dev, int idx) +{ + g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); + int r; + + r =3D event_notifier_init(&svq->kick_notifier, 0); + if (r !=3D 0) { + error_report("Couldn't create kick event notifier: %s", + strerror(errno)); + goto err_init_kick_notifier; + } + + r =3D event_notifier_init(&svq->call_notifier, 0); + if (r !=3D 0) { + error_report("Couldn't create call event notifier: %s", + strerror(errno)); + goto err_init_call_notifier; + } + + return g_steal_pointer(&svq); + +err_init_call_notifier: + event_notifier_cleanup(&svq->kick_notifier); + +err_init_kick_notifier: + return NULL; +} + +/* + * Free the resources of the shadow virtqueue. + */ +void vhost_shadow_vq_free(VhostShadowVirtqueue *vq) +{ + event_notifier_cleanup(&vq->kick_notifier); + event_notifier_cleanup(&vq->call_notifier); + g_free(vq); +} diff --git a/hw/virtio/meson.build b/hw/virtio/meson.build index fbff9bc9d4..8b5a0225fe 100644 --- a/hw/virtio/meson.build +++ b/hw/virtio/meson.build @@ -11,7 +11,7 @@ softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost-= stub.c')) =20 virtio_ss =3D ss.source_set() virtio_ss.add(files('virtio.c')) -virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backe= nd.c')) +virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backe= nd.c', 'vhost-shadow-virtqueue.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER', if_true: files('vhost-user.c')) virtio_ss.add(when: 'CONFIG_VHOST_VDPA', if_true: files('vhost-vdpa.c')) virtio_ss.add(when: 'CONFIG_VIRTIO_BALLOON', if_true: files('virtio-balloo= n.c')) --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442116; cv=none; d=zohomail.com; s=zohoarc; b=JPrjLni0KjUw2el80Kf7eemI6zUtu0TbweYPg8X21trscch0pP8mqpQbnNHuZEFUvrG156QJCF0xF3sigaE8lEywHZdFmlJ3spNOJTSbNhY1SG+hGToNw5BN5EUBkDqQLKjBTnD6JlIO6pYBBPiEytDHyrpqukyIKxL3yxHm6cY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442116; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=VetRo+0BKe9xyEK8yuAfA4AJjKiZmn+zmiMLp+Fzc74=; b=SaBWOd8lkZ801OTPgN7HJI8aTBvNvL81IXCQwniaKKBi1F/FbVKSp4DIX7FqGsobksC8tEm/lqPiSW4/vxbyuDlOhwpH/nWx1iJX10YYrI+1pd17dgjWMM7LHlEtqAcQ1nhhZSmphKpuf3IekpdBHyq7OI6Fg121sK2Px20f0FE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 162144211687154.73164540264554; Wed, 19 May 2021 09:35:16 -0700 (PDT) Received: from localhost ([::1]:36500 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPAB-000235-Ro for importer@patchew.org; Wed, 19 May 2021 12:35:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34354) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP50-0002Tj-2g for qemu-devel@nongnu.org; Wed, 19 May 2021 12:29:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55632) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP4x-0005CO-4t for qemu-devel@nongnu.org; Wed, 19 May 2021 12:29:53 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-136-1FGdS7b7P2iYSOKUeJoUjw-1; Wed, 19 May 2021 12:29:47 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3530F801106; Wed, 19 May 2021 16:29:46 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 74DF35D6AC; Wed, 19 May 2021 16:29:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441790; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VetRo+0BKe9xyEK8yuAfA4AJjKiZmn+zmiMLp+Fzc74=; b=fFPurX1hen2LsZ8cUrZ5mgIVRn5vnfb4HdA9CpA+BR95x4fB8PllZEub5YA5hqyDDFRmB4 PBJ6ZULr5bGucWZhp/3/3tokJM0E1W29aRlgbKvoY8Pb+JdCyiVzZiNzAM0gjPIaxMtwW2 zXDCOOoidUbSG4NbOMclOeNbmFtdetg= X-MC-Unique: 1FGdS7b7P2iYSOKUeJoUjw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 04/29] vhost: Add x-vhost-enable-shadow-vq qmp Date: Wed, 19 May 2021 18:28:38 +0200 Message-Id: <20210519162903.1172366-5-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Command to enable shadow virtqueue looks like: { "execute": "x-vhost-enable-shadow-vq", "arguments": { "name": "dev0", "enable": true } } Signed-off-by: Eugenio P=C3=A9rez --- qapi/net.json | 22 ++++++++++++++++++++++ hw/virtio/vhost.c | 6 ++++++ 2 files changed, 28 insertions(+) diff --git a/qapi/net.json b/qapi/net.json index c31748c87f..660feafdd2 100644 --- a/qapi/net.json +++ b/qapi/net.json @@ -77,6 +77,28 @@ ## { 'command': 'netdev_del', 'data': {'id': 'str'} } =20 +## +# @x-vhost-enable-shadow-vq: +# +# Use vhost shadow virtqueue. +# +# @name: the device name of the VirtIO device +# +# @enable: true to use he alternate shadow VQ notification path +# +# Returns: Error if failure, or 'no error' for success. Not found if vhost= is not enabled. +# +# Since: 6.1 +# +# Example: +# +# -> { "execute": "x-vhost-enable-shadow-vq", "arguments": { "name": "virt= io-net", "enable": false } } +# +## +{ 'command': 'x-vhost-enable-shadow-vq', + 'data': {'name': 'str', 'enable': 'bool'}, + 'if': 'defined(CONFIG_VHOST_KERNEL)' } + ## # @NetLegacyNicOptions: # diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 40f9f64ebd..c4c1f80661 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -15,6 +15,7 @@ =20 #include "qemu/osdep.h" #include "qapi/error.h" +#include "qapi/qapi-commands-net.h" #include "hw/virtio/vhost.h" #include "qemu/atomic.h" #include "qemu/range.h" @@ -1831,3 +1832,8 @@ int vhost_net_set_backend(struct vhost_dev *hdev, =20 return -1; } + +void qmp_x_vhost_enable_shadow_vq(const char *name, bool enable, Error **e= rrp) +{ + error_setg(errp, "Shadow virtqueue still not implemented"); +} --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442135; cv=none; d=zohomail.com; s=zohoarc; b=WqfhCQaMYbWoNWJH0pLp/rCAxvOtkuh9DkNUlYLn/036ojRZfok1l19vf75JiUgacN0GFBr5wiL6nD+2X0nL3xLws3FposoBRB8NYmOctjTeiQ9N3oqFlJrDvbRFWoLPxJUasr2my++O5A9uxTAOz7gLiuKTsyZQJwFSZyhlgeg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442135; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lN9E94Er5x6d1jL2uZjeSkSsMx9OR6z4PGUDVqDULLg=; b=UoQeVuSQNsoIV9YSlF01p5oKkz1ytMR4yWgxbsw0I6xghWGepAOMKctfEFyoG4pstdoBUFx6Zb8fth0JB7q/DQwVb0Sx31+g1QwkE7swDuFQe+Qwn4MLcbX8hGDO4k+JWWYWDFllJ8x8UJ1nnQkvyewG3wvtCrWr1Os3Iy7DztQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442135318585.2028981405134; Wed, 19 May 2021 09:35:35 -0700 (PDT) Received: from localhost ([::1]:37176 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPAU-0002Wp-6V for importer@patchew.org; Wed, 19 May 2021 12:35:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34368) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP54-0002Ub-H4 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:29:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:39417) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP50-0005Cd-Ew for qemu-devel@nongnu.org; Wed, 19 May 2021 12:29:57 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-405-s9GvOcEjMEOa3Uv9TLvyJg-1; Wed, 19 May 2021 12:29:51 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2223A801817; Wed, 19 May 2021 16:29:50 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9355B5D6AC; Wed, 19 May 2021 16:29:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441793; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lN9E94Er5x6d1jL2uZjeSkSsMx9OR6z4PGUDVqDULLg=; b=MeM2BPLexVJWQTb8wkp/iNvsqQWrSULFJvX+9UKcaqvEnW7WovnNb1F+xdVXRpfzYmCv/l 9vnRmnncrk2tSTRKJJrVX9S2BjX+1eTAwt3H4fxEbKsydKT0NjpHkJp93AYBwW0u7NeLZQ UNlHWRFwNcwVjbxmUTYU5c16mM8IUwM= X-MC-Unique: s9GvOcEjMEOa3Uv9TLvyJg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 05/29] virtio: Add VIRTIO_F_QUEUE_STATE Date: Wed, 19 May 2021 18:28:39 +0200 Message-Id: <20210519162903.1172366-6-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Implementation of RFC of device state capability: https://lists.oasis-open.org/archives/virtio-comment/202012/msg00005.html With this capability, vdpa device can reset it's index so it can start consuming from shadow virtqueue (SVQ), that start with state 0. Another approach would be to make SVQ to start forwarding from the state of the device when the later is stopped, but this device capability is needed at the destination of live migration anyway. The use case is to test SVQ with virtio-pci vdpa (vp_vdpa) with nested virtualization. Spawning a L0 qemu with a virtio-net device, use vp_vdpa driver to handle it in the guest, and then spawn a L1 qemu using that vdpa device. When L1 qemu calls device to set a new state though vdpa ioctl, vp_vdpa should set each queue state though virtio VIRTIO_PCI_COMMON_Q_AVAIL_STATE. Since this is only for testing vhost-vdpa, it's added here before of proposing to kernel code. No effort is done for checking that device can actually change its state, its layout, or if the device even supports to change state at all. These will be added in the future. Also, a modified version of vp_vdpa that allows to set these in PCI config is needed. TODO: Check for feature enabled and split in virtio pci config Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/virtio-pci.h | 1 + include/hw/virtio/virtio.h | 4 +++- include/standard-headers/linux/virtio_config.h | 3 +++ include/standard-headers/linux/virtio_pci.h | 2 ++ hw/virtio/virtio-pci.c | 9 +++++++++ 5 files changed, 18 insertions(+), 1 deletion(-) diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h index d7d5d403a9..69e34449cd 100644 --- a/hw/virtio/virtio-pci.h +++ b/hw/virtio/virtio-pci.h @@ -115,6 +115,7 @@ typedef struct VirtIOPCIQueue { uint32_t desc[2]; uint32_t avail[2]; uint32_t used[2]; + uint16_t state; } VirtIOPCIQueue; =20 struct VirtIOPCIProxy { diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index c2c7cee993..dfcc7d8350 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -289,7 +289,9 @@ typedef struct VirtIORNGConf VirtIORNGConf; DEFINE_PROP_BIT64("iommu_platform", _state, _field, \ VIRTIO_F_IOMMU_PLATFORM, false), \ DEFINE_PROP_BIT64("packed", _state, _field, \ - VIRTIO_F_RING_PACKED, false) + VIRTIO_F_RING_PACKED, false), \ + DEFINE_PROP_BIT64("save_restore_q_state", _state, _field, \ + VIRTIO_F_QUEUE_STATE, true) =20 hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n); bool virtio_queue_enabled_legacy(VirtIODevice *vdev, int n); diff --git a/include/standard-headers/linux/virtio_config.h b/include/stand= ard-headers/linux/virtio_config.h index 22e3a85f67..59fad3eb45 100644 --- a/include/standard-headers/linux/virtio_config.h +++ b/include/standard-headers/linux/virtio_config.h @@ -90,4 +90,7 @@ * Does the device support Single Root I/O Virtualization? */ #define VIRTIO_F_SR_IOV 37 + +/* Device support save and restore virtqueue state */ +#define VIRTIO_F_QUEUE_STATE 40 #endif /* _LINUX_VIRTIO_CONFIG_H */ diff --git a/include/standard-headers/linux/virtio_pci.h b/include/standard= -headers/linux/virtio_pci.h index db7a8e2fcb..c8d9802a87 100644 --- a/include/standard-headers/linux/virtio_pci.h +++ b/include/standard-headers/linux/virtio_pci.h @@ -164,6 +164,7 @@ struct virtio_pci_common_cfg { uint32_t queue_avail_hi; /* read-write */ uint32_t queue_used_lo; /* read-write */ uint32_t queue_used_hi; /* read-write */ + uint16_t queue_avail_state; /* read-write */ }; =20 /* Fields in VIRTIO_PCI_CAP_PCI_CFG: */ @@ -202,6 +203,7 @@ struct virtio_pci_cfg_cap { #define VIRTIO_PCI_COMMON_Q_AVAILHI 44 #define VIRTIO_PCI_COMMON_Q_USEDLO 48 #define VIRTIO_PCI_COMMON_Q_USEDHI 52 +#define VIRTIO_PCI_COMMON_Q_AVAIL_STATE 56 =20 #endif /* VIRTIO_PCI_NO_MODERN */ =20 diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 883045a223..ddb6fff098 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -1216,6 +1216,9 @@ static uint64_t virtio_pci_common_read(void *opaque, = hwaddr addr, case VIRTIO_PCI_COMMON_Q_USEDHI: val =3D proxy->vqs[vdev->queue_sel].used[1]; break; + case VIRTIO_PCI_COMMON_Q_AVAIL_STATE: + val =3D virtio_queue_get_last_avail_idx(vdev, vdev->queue_sel); + break; default: val =3D 0; } @@ -1298,6 +1301,8 @@ static void virtio_pci_common_write(void *opaque, hwa= ddr addr, proxy->vqs[vdev->queue_sel].avail[0], ((uint64_t)proxy->vqs[vdev->queue_sel].used[1]) << = 32 | proxy->vqs[vdev->queue_sel].used[0]); + virtio_queue_set_last_avail_idx(vdev, vdev->queue_sel, + proxy->vqs[vdev->queue_sel].state); proxy->vqs[vdev->queue_sel].enabled =3D 1; } else { virtio_error(vdev, "wrong value for queue_enable %"PRIx64, val= ); @@ -1321,6 +1326,9 @@ static void virtio_pci_common_write(void *opaque, hwa= ddr addr, case VIRTIO_PCI_COMMON_Q_USEDHI: proxy->vqs[vdev->queue_sel].used[1] =3D val; break; + case VIRTIO_PCI_COMMON_Q_AVAIL_STATE: + proxy->vqs[vdev->queue_sel].state =3D val; + break; default: break; } @@ -1900,6 +1908,7 @@ static void virtio_pci_reset(DeviceState *qdev) proxy->vqs[i].desc[0] =3D proxy->vqs[i].desc[1] =3D 0; proxy->vqs[i].avail[0] =3D proxy->vqs[i].avail[1] =3D 0; proxy->vqs[i].used[0] =3D proxy->vqs[i].used[1] =3D 0; + proxy->vqs[i].state =3D 0; } =20 if (pci_is_express(dev)) { --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442287; cv=none; d=zohomail.com; s=zohoarc; b=cNuGRpt+sSfGiMjOT8TJU0y10DR/DX5VW7oIiWGwoFIaE0qYtyeI2pQ2tlZ8vXUDEMgeo+Yr19NC4IOKVW/MA/FuiydyZTfucdnykOz08fvmGX2cr4wPBBcFaF14KWIAVsh1cKxDRBb2ZTHQDNdpa/DDehy2Y4EwfpSuyP8D2gA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442287; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=6l3SqLSeKmEhV1NXWaESuTSmX7euR+x4nuSnTsSY50I=; b=Td56FrWmb3w7VBzrLmqtlbRzXOtPRC6goK2KsAjHNt+Dwk3kJ0cjhboPkKky8Wuc7QNAML+WLeVLi7raMCApDy21aM5u4+NWjHuMYKXsI4+2mzN71pg7nI6X2st6kFrqImQ78wgiSzMfP7e4Z9XAbmKoglLqIycI+vReunVH5ac= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442287705652.1802917999851; Wed, 19 May 2021 09:38:07 -0700 (PDT) Received: from localhost ([::1]:45802 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPCw-0008RZ-Hw for importer@patchew.org; Wed, 19 May 2021 12:38:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34404) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP58-0002hp-NK for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:52561) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP56-0005EW-Pv for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:02 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-514-K64JIjBIN6qxux0KWwexJQ-1; Wed, 19 May 2021 12:29:58 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E8865180FD65; Wed, 19 May 2021 16:29:56 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 823045D6AC; Wed, 19 May 2021 16:29:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441800; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6l3SqLSeKmEhV1NXWaESuTSmX7euR+x4nuSnTsSY50I=; b=SIaMc5dU4+GB7ko04IJlMa2yGZVl4h0RLpdR/4JgMcNljBr6uDf3OA0E6v1GN2Jinwc7pk 7T/1+g1IWJe60lxkhi84hciah5rnYJL94pTwS42fWberd8II9Oum9KBSn89jObbKWLO29z CgtXatjbL3rHj6x90hRuOlfpA2+ZxiQ= X-MC-Unique: K64JIjBIN6qxux0KWwexJQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 06/29] virtio-net: Honor VIRTIO_CONFIG_S_DEVICE_STOPPED Date: Wed, 19 May 2021 18:28:40 +0200 Message-Id: <20210519162903.1172366-7-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) So the guest can stop and start net device. It implements the RFC https://lists.oasis-open.org/archives/virtio-comment/202012/msg00027.html To stop (as "pause") the device is required to migrate status and vring addresses between device and SVQ. This is a WIP commit: as with VIRTIO_F_QUEUE_STATE, is introduced in virtio_config.h before of even proposing for the kernel, with no feature flag, and, with no checking in the device. It also needs a modified vp_vdpa driver that supports to set and retrieve status. Signed-off-by: Eugenio P=C3=A9rez --- include/standard-headers/linux/virtio_config.h | 2 ++ hw/net/virtio-net.c | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/include/standard-headers/linux/virtio_config.h b/include/stand= ard-headers/linux/virtio_config.h index 59fad3eb45..b3f6b1365d 100644 --- a/include/standard-headers/linux/virtio_config.h +++ b/include/standard-headers/linux/virtio_config.h @@ -40,6 +40,8 @@ #define VIRTIO_CONFIG_S_DRIVER_OK 4 /* Driver has finished configuring features */ #define VIRTIO_CONFIG_S_FEATURES_OK 8 +/* Device is stopped */ +#define VIRTIO_CONFIG_S_DEVICE_STOPPED 32 /* Device entered invalid state, driver must reset it */ #define VIRTIO_CONFIG_S_NEEDS_RESET 0x40 /* We've given up on this device. */ diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index 96a3cc8357..2d3caea289 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -198,7 +198,9 @@ static bool virtio_net_started(VirtIONet *n, uint8_t st= atus) { VirtIODevice *vdev =3D VIRTIO_DEVICE(n); return (status & VIRTIO_CONFIG_S_DRIVER_OK) && - (n->status & VIRTIO_NET_S_LINK_UP) && vdev->vm_running; + (!(n->status & VIRTIO_CONFIG_S_DEVICE_STOPPED)) && + (n->status & VIRTIO_NET_S_LINK_UP) && + vdev->vm_running; } =20 static void virtio_net_announce_notify(VirtIONet *net) --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442156; cv=none; d=zohomail.com; s=zohoarc; b=S6Eb4GZtITheVzz1IeBh35C3Bl6BMwunGUs+c36DWfOMARNcPpVIRMWtCWQMB/BpAPxfrvxQnsBJSA2/yJbQ3MwM2sUfkvai4kI+x482ForrYL2vRWvYyo2u0oXYSxkblPTQwJE6jrouMyZJstqAtdIzkF0hLiVDEWThWtBcaKs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442156; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=/nDkxBDtSYmLMz+5vg4uCbwepfBk8wkLkW97MdIYV4Y=; b=QMha1GJIEnA1HbiJaQ7eLvVcUBgLyu82cH3bDknuAkH0ERAKGWQVbpqOC/xFzCBZ0Y3o/NomCYeStnbkWlqGGNegcfsSpdXC9CTybP0p0hcuD4lMKdRZ9t9xEt6De+LPR6Kz9k6aanTtl2V9909rhSe0NS5BunFujz8sHxd8yUs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442156577806.8526328137655; Wed, 19 May 2021 09:35:56 -0700 (PDT) Received: from localhost ([::1]:37650 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPAo-0002pT-DV for importer@patchew.org; Wed, 19 May 2021 12:35:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34458) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP5H-00037l-B3 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46514) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP5E-0005Ix-LA for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:11 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-61-nflRJF1DNla5xrJ-BBYlxQ-1; Wed, 19 May 2021 12:30:06 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8DCA7180FD69; Wed, 19 May 2021 16:30:04 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 569AB5D6D5; Wed, 19 May 2021 16:29:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/nDkxBDtSYmLMz+5vg4uCbwepfBk8wkLkW97MdIYV4Y=; b=D8L7JTY3ssURct4a8YndakfxlUmax8hGS2t40Qqp97L5jshHVcFZztLeQwaEtixb5I0+9o h698G9sVqsvqu4LeW3rZiL8wHD6WPpWR4V4FXk7w8jhqUndVrTw2zJpO1nexvG5DJMOwEp DQiUeXWcdIQd3MwItyTV2VylYpE0M7E= X-MC-Unique: nflRJF1DNla5xrJ-BBYlxQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 07/29] vhost: Route guest->host notification through shadow virtqueue Date: Wed, 19 May 2021 18:28:41 +0200 Message-Id: <20210519162903.1172366-8-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Shadow virtqueue notifications forwarding is disabled when vhost_dev stops, so code flow follows usual cleanup. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 7 ++ include/hw/virtio/vhost.h | 4 + hw/virtio/vhost-shadow-virtqueue.c | 113 ++++++++++++++++++++++- hw/virtio/vhost.c | 143 ++++++++++++++++++++++++++++- 4 files changed, 265 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 6cc18d6acb..c891c6510d 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -17,6 +17,13 @@ =20 typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 +bool vhost_shadow_vq_start(struct vhost_dev *dev, + unsigned idx, + VhostShadowVirtqueue *svq); +void vhost_shadow_vq_stop(struct vhost_dev *dev, + unsigned idx, + VhostShadowVirtqueue *svq); + VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_dev *dev, int idx); =20 void vhost_shadow_vq_free(VhostShadowVirtqueue *vq); diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index ac963bf23d..7ffdf9aea0 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -55,6 +55,8 @@ struct vhost_iommu { QLIST_ENTRY(vhost_iommu) iommu_next; }; =20 +typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; + typedef struct VhostDevConfigOps { /* Vhost device config space changed callback */ @@ -83,7 +85,9 @@ struct vhost_dev { uint64_t backend_cap; bool started; bool log_enabled; + bool shadow_vqs_enabled; uint64_t log_size; + VhostShadowVirtqueue **shadow_vqs; Error *migration_blocker; const VhostOps *vhost_ops; void *opaque; diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 4512e5b058..3e43399e9c 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -8,9 +8,12 @@ */ =20 #include "hw/virtio/vhost-shadow-virtqueue.h" +#include "hw/virtio/vhost.h" + +#include "standard-headers/linux/vhost_types.h" =20 #include "qemu/error-report.h" -#include "qemu/event_notifier.h" +#include "qemu/main-loop.h" =20 /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { @@ -18,14 +21,121 @@ typedef struct VhostShadowVirtqueue { EventNotifier kick_notifier; /* Shadow call notifier, sent to vhost */ EventNotifier call_notifier; + + /* + * Borrowed virtqueue's guest to host notifier. + * To borrow it in this event notifier allows to register on the event + * loop and access the associated shadow virtqueue easily. If we use t= he + * VirtQueue, we don't have an easy way to retrieve it. + * + * So shadow virtqueue must not clean it, or we would lose VirtQueue o= ne. + */ + EventNotifier host_notifier; + + /* Virtio queue shadowing */ + VirtQueue *vq; } VhostShadowVirtqueue; =20 +/* Forward guest notifications */ +static void vhost_handle_guest_kick(EventNotifier *n) +{ + VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, + host_notifier); + + if (unlikely(!event_notifier_test_and_clear(n))) { + return; + } + + event_notifier_set(&svq->kick_notifier); +} + +/* + * Restore the vhost guest to host notifier, i.e., disables svq effect. + */ +static int vhost_shadow_vq_restore_vdev_host_notifier(struct vhost_dev *de= v, + unsigned vhost_index, + VhostShadowVirtqueue = *svq) +{ + EventNotifier *vq_host_notifier =3D virtio_queue_get_host_notifier(svq= ->vq); + struct vhost_vring_file file =3D { + .index =3D vhost_index, + .fd =3D event_notifier_get_fd(vq_host_notifier), + }; + int r; + + /* Restore vhost kick */ + r =3D dev->vhost_ops->vhost_set_vring_kick(dev, &file); + return r ? -errno : 0; +} + +/* + * Start shadow virtqueue operation. + * @dev vhost device + * @hidx vhost virtqueue index + * @svq Shadow Virtqueue + */ +bool vhost_shadow_vq_start(struct vhost_dev *dev, + unsigned idx, + VhostShadowVirtqueue *svq) +{ + EventNotifier *vq_host_notifier =3D virtio_queue_get_host_notifier(svq= ->vq); + struct vhost_vring_file file =3D { + .index =3D idx, + .fd =3D event_notifier_get_fd(&svq->kick_notifier), + }; + int r; + + /* Check that notifications are still going directly to vhost dev */ + assert(virtio_queue_is_host_notifier_enabled(svq->vq)); + + /* + * event_notifier_set_handler already checks for guest's notifications= if + * they arrive in the switch, so there is no need to explicitely check= for + * them. + */ + event_notifier_init_fd(&svq->host_notifier, + event_notifier_get_fd(vq_host_notifier)); + event_notifier_set_handler(&svq->host_notifier, vhost_handle_guest_kic= k); + + r =3D dev->vhost_ops->vhost_set_vring_kick(dev, &file); + if (unlikely(r !=3D 0)) { + error_report("Couldn't set kick fd: %s", strerror(errno)); + goto err_set_vring_kick; + } + + return true; + +err_set_vring_kick: + event_notifier_set_handler(&svq->host_notifier, NULL); + + return false; +} + +/* + * Stop shadow virtqueue operation. + * @dev vhost device + * @idx vhost queue index + * @svq Shadow Virtqueue + */ +void vhost_shadow_vq_stop(struct vhost_dev *dev, + unsigned idx, + VhostShadowVirtqueue *svq) +{ + int r =3D vhost_shadow_vq_restore_vdev_host_notifier(dev, idx, svq); + if (unlikely(r < 0)) { + error_report("Couldn't restore vq kick fd: %s", strerror(-r)); + } + + event_notifier_set_handler(&svq->host_notifier, NULL); +} + /* * Creates vhost shadow virtqueue, and instruct vhost device to use the sh= adow * methods and file descriptors. */ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_dev *dev, int idx) { + int vq_idx =3D dev->vq_index + idx; g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); int r; =20 @@ -43,6 +153,7 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_d= ev *dev, int idx) goto err_init_call_notifier; } =20 + svq->vq =3D virtio_get_queue(dev->vdev, vq_idx); return g_steal_pointer(&svq); =20 err_init_call_notifier: diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index c4c1f80661..84091b5251 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -25,6 +25,7 @@ #include "exec/address-spaces.h" #include "hw/virtio/virtio-bus.h" #include "hw/virtio/virtio-access.h" +#include "hw/virtio/vhost-shadow-virtqueue.h" #include "migration/blocker.h" #include "migration/qemu-file-types.h" #include "sysemu/dma.h" @@ -1219,6 +1220,74 @@ static void vhost_virtqueue_stop(struct vhost_dev *d= ev, 0, virtio_queue_get_desc_size(vdev, idx)); } =20 +static int vhost_sw_live_migration_stop(struct vhost_dev *dev) +{ + int idx; + + dev->shadow_vqs_enabled =3D false; + + for (idx =3D 0; idx < dev->nvqs; ++idx) { + vhost_shadow_vq_stop(dev, idx, dev->shadow_vqs[idx]); + vhost_shadow_vq_free(dev->shadow_vqs[idx]); + } + + g_free(dev->shadow_vqs); + dev->shadow_vqs =3D NULL; + return 0; +} + +static int vhost_sw_live_migration_start(struct vhost_dev *dev) +{ + int idx, stop_idx; + + dev->shadow_vqs =3D g_new0(VhostShadowVirtqueue *, dev->nvqs); + for (idx =3D 0; idx < dev->nvqs; ++idx) { + dev->shadow_vqs[idx] =3D vhost_shadow_vq_new(dev, idx); + if (unlikely(dev->shadow_vqs[idx] =3D=3D NULL)) { + goto err_new; + } + } + + dev->shadow_vqs_enabled =3D true; + for (idx =3D 0; idx < dev->nvqs; ++idx) { + bool ok =3D vhost_shadow_vq_start(dev, idx, dev->shadow_vqs[idx]); + if (unlikely(!ok)) { + goto err_start; + } + } + + return 0; + +err_start: + dev->shadow_vqs_enabled =3D false; + for (stop_idx =3D 0; stop_idx < idx; stop_idx++) { + vhost_shadow_vq_stop(dev, idx, dev->shadow_vqs[stop_idx]); + } + +err_new: + for (idx =3D 0; idx < dev->nvqs; ++idx) { + vhost_shadow_vq_free(dev->shadow_vqs[idx]); + } + g_free(dev->shadow_vqs); + + return -1; +} + +static int vhost_sw_live_migration_enable(struct vhost_dev *dev, + bool enable_lm) +{ + int r; + + if (enable_lm =3D=3D dev->shadow_vqs_enabled) { + return 0; + } + + r =3D enable_lm ? vhost_sw_live_migration_start(dev) + : vhost_sw_live_migration_stop(dev); + + return r; +} + static void vhost_eventfd_add(MemoryListener *listener, MemoryRegionSection *section, bool match_data, uint64_t data, EventNotifie= r *e) @@ -1381,6 +1450,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaq= ue, hdev->log =3D NULL; hdev->log_size =3D 0; hdev->log_enabled =3D false; + hdev->shadow_vqs_enabled =3D false; hdev->started =3D false; memory_listener_register(&hdev->memory_listener, &address_space_memory= ); QLIST_INSERT_HEAD(&vhost_devices, hdev, entry); @@ -1484,6 +1554,10 @@ void vhost_dev_disable_notifiers(struct vhost_dev *h= dev, VirtIODevice *vdev) BusState *qbus =3D BUS(qdev_get_parent_bus(DEVICE(vdev))); int i, r; =20 + if (hdev->shadow_vqs_enabled) { + vhost_sw_live_migration_enable(hdev, false); + } + for (i =3D 0; i < hdev->nvqs; ++i) { r =3D virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), hdev->vq_inde= x + i, false); @@ -1798,6 +1872,7 @@ fail_features: void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev) { int i; + bool is_shadow_vqs_enabled =3D hdev->shadow_vqs_enabled; =20 /* should only be called after backend is connected */ assert(hdev->vhost_ops); @@ -1805,7 +1880,16 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODe= vice *vdev) if (hdev->vhost_ops->vhost_dev_start) { hdev->vhost_ops->vhost_dev_start(hdev, false); } + if (is_shadow_vqs_enabled) { + /* Shadow virtqueue will be stopped */ + hdev->shadow_vqs_enabled =3D false; + } for (i =3D 0; i < hdev->nvqs; ++i) { + if (is_shadow_vqs_enabled) { + vhost_shadow_vq_stop(hdev, i, hdev->shadow_vqs[i]); + vhost_shadow_vq_free(hdev->shadow_vqs[i]); + } + vhost_virtqueue_stop(hdev, vdev, hdev->vqs + i, @@ -1819,6 +1903,8 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODev= ice *vdev) memory_listener_unregister(&hdev->iommu_listener); } vhost_log_put(hdev, true); + g_free(hdev->shadow_vqs); + hdev->shadow_vqs_enabled =3D false; hdev->started =3D false; hdev->vdev =3D NULL; } @@ -1835,5 +1921,60 @@ int vhost_net_set_backend(struct vhost_dev *hdev, =20 void qmp_x_vhost_enable_shadow_vq(const char *name, bool enable, Error **e= rrp) { - error_setg(errp, "Shadow virtqueue still not implemented"); + struct vhost_dev *hdev, *hdev_err; + VirtIODevice *vdev; + const char *err_cause =3D NULL; + int r; + ErrorClass err_class =3D ERROR_CLASS_GENERIC_ERROR; + + QLIST_FOREACH(hdev, &vhost_devices, entry) { + if (hdev->vdev && 0 =3D=3D strcmp(hdev->vdev->name, name)) { + vdev =3D hdev->vdev; + break; + } + } + + if (!hdev) { + err_class =3D ERROR_CLASS_DEVICE_NOT_FOUND; + err_cause =3D "Device not found"; + goto not_found_err; + } + + for ( ; hdev; hdev =3D QLIST_NEXT(hdev, entry)) { + if (vdev !=3D hdev->vdev) { + continue; + } + + if (!hdev->started) { + err_cause =3D "Device is not started"; + goto err; + } + + r =3D vhost_sw_live_migration_enable(hdev, enable); + if (unlikely(r)) { + err_cause =3D "Error enabling (see monitor)"; + goto err; + } + } + + return; + +err: + QLIST_FOREACH(hdev_err, &vhost_devices, entry) { + if (hdev_err =3D=3D hdev) { + break; + } + + if (vdev !=3D hdev->vdev) { + continue; + } + + vhost_sw_live_migration_enable(hdev, !enable); + } + +not_found_err: + if (err_cause) { + error_set(errp, err_class, + "Can't enable shadow vq on %s: %s", name, err_cause); + } } --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442117; cv=none; d=zohomail.com; s=zohoarc; b=XAjq6zlRXQXlXS3B/Sj6A/RcWcyeYtkyiYX39h24rnRqZi2Vw5jt3ElctE8RW2umt0zXkgwX6bW6dgXnM+Iym3gGjLrh6GSn8iNA/mPre5CJGebfgdkJeduKYq8IdB1pZWc6tvzPlJMBLI6FqZO3sYIAK2xAlww18bwpcSClntE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442117; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wndRUAl0yqcDZHg+lrz+7s4nniC0MTgzWuOP5HXK5QE=; b=Mvk5Rvu2IgTRbH4CYak2mhZQNEj4mF7afpMsCYCOZ3BvVM94pIaxTKgEF1iI5iwkbuLVidMZuApqTae1NZ/FZvpzc0IhwNwSU5lkM6kt1nOSzpEhd3s54LjgRgQdDpHF/OvlJu51Va1A2Bl5y15yA+fyZKoxK4zOVwpPMfNWI0k= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442117615610.6279337485627; Wed, 19 May 2021 09:35:17 -0700 (PDT) Received: from localhost ([::1]:36536 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPAC-00024T-Iu for importer@patchew.org; Wed, 19 May 2021 12:35:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34492) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP5L-0003B3-C6 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43368) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP5H-0005Jw-Ka for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:15 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-324-RCiFpXrDOLO-U8uNDQBAxA-1; Wed, 19 May 2021 12:30:09 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 80EB0100747B; Wed, 19 May 2021 16:30:08 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id EDC1A5D6AC; Wed, 19 May 2021 16:30:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441811; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wndRUAl0yqcDZHg+lrz+7s4nniC0MTgzWuOP5HXK5QE=; b=OvUIwfMy5i/9UaCPAX4RK73yO49Y27ButvR3Ib+ALCwwrqc7gW6j/SxSVjqzGtJgqjMoVG utDKMSlHMDBQEM2VBI+f3Atlb1CMDvNVqJpvKC9blFl0YfBVjdpgroQxhtMa+fz6NwJ/vL m3zF6i/Xqgn/bIXWx/GyI26mNvnuxQ0= X-MC-Unique: RCiFpXrDOLO-U8uNDQBAxA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 08/29] vhost: Route host->guest notification through shadow virtqueue Date: Wed, 19 May 2021 18:28:42 +0200 Message-Id: <20210519162903.1172366-9-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 3 + include/hw/virtio/vhost.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 95 ++++++++++++++++++++++++++++++ hw/virtio/vhost.c | 15 +++++ 4 files changed, 114 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index c891c6510d..2ca4b92b12 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -17,6 +17,9 @@ =20 typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 +void vhost_shadow_vq_mask(VhostShadowVirtqueue *svq, EventNotifier *masked= ); +void vhost_shadow_vq_unmask(VhostShadowVirtqueue *svq); + bool vhost_shadow_vq_start(struct vhost_dev *dev, unsigned idx, VhostShadowVirtqueue *svq); diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 7ffdf9aea0..67cedf83da 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -28,6 +28,7 @@ struct vhost_virtqueue { unsigned avail_size; unsigned long long used_phys; unsigned used_size; + /* Access/writing to notifier_is_masked is protected by BQL */ bool notifier_is_masked; EventNotifier masked_notifier; struct vhost_dev *dev; diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 3e43399e9c..7d76e271a5 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -32,8 +32,16 @@ typedef struct VhostShadowVirtqueue { */ EventNotifier host_notifier; =20 + /* (Possible) masked notifier */ + struct { + EventNotifier *n; + } masked_notifier; + /* Virtio queue shadowing */ VirtQueue *vq; + + /* Virtio device */ + VirtIODevice *vdev; } VhostShadowVirtqueue; =20 /* Forward guest notifications */ @@ -49,6 +57,58 @@ static void vhost_handle_guest_kick(EventNotifier *n) event_notifier_set(&svq->kick_notifier); } =20 +/* Forward vhost notifications */ +static void vhost_shadow_vq_handle_call_no_test(EventNotifier *n) +{ + VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, + call_notifier); + EventNotifier *masked_notifier; + + masked_notifier =3D svq->masked_notifier.n; + + if (!masked_notifier) { + unsigned n =3D virtio_get_queue_index(svq->vq); + virtio_queue_invalidate_signalled_used(svq->vdev, n); + virtio_notify_irqfd(svq->vdev, svq->vq); + } else { + event_notifier_set(svq->masked_notifier.n); + } +} + +static void vhost_shadow_vq_handle_call(EventNotifier *n) +{ + if (likely(event_notifier_test_and_clear(n))) { + vhost_shadow_vq_handle_call_no_test(n); + } +} + +/* + * Mask the shadow virtqueue. + * + * It can be called from a guest masking vmexit or shadow virtqueue start + * through QMP. + * + * @vq Shadow virtqueue + * @masked Masked notifier to signal instead of guest + */ +void vhost_shadow_vq_mask(VhostShadowVirtqueue *svq, EventNotifier *masked) +{ + svq->masked_notifier.n =3D masked; +} + +/* + * Unmask the shadow virtqueue. + * + * It can be called from a guest unmasking vmexit or shadow virtqueue start + * through QMP. + * + * @vq Shadow virtqueue + */ +void vhost_shadow_vq_unmask(VhostShadowVirtqueue *svq) +{ + svq->masked_notifier.n =3D NULL; +} + /* * Restore the vhost guest to host notifier, i.e., disables svq effect. */ @@ -103,8 +163,33 @@ bool vhost_shadow_vq_start(struct vhost_dev *dev, goto err_set_vring_kick; } =20 + /* Set vhost call */ + file.fd =3D event_notifier_get_fd(&svq->call_notifier), + r =3D dev->vhost_ops->vhost_set_vring_call(dev, &file); + if (unlikely(r !=3D 0)) { + error_report("Couldn't set call fd: %s", strerror(errno)); + goto err_set_vring_call; + } + + /* Set shadow vq -> guest notifier */ + assert(dev->shadow_vqs_enabled); + vhost_virtqueue_mask(dev, dev->vdev, dev->vq_index + idx, + dev->vqs[idx].notifier_is_masked); + + if (dev->vqs[idx].notifier_is_masked && + event_notifier_test_and_clear(&dev->vqs[idx].masked_notifie= r)) { + /* Check for pending notifications from the device */ + vhost_shadow_vq_handle_call_no_test(&svq->call_notifier); + } + return true; =20 +err_set_vring_call: + r =3D vhost_shadow_vq_restore_vdev_host_notifier(dev, idx, svq); + if (unlikely(r < 0)) { + error_report("Couldn't restore vq kick fd: %s", strerror(-r)); + } + err_set_vring_kick: event_notifier_set_handler(&svq->host_notifier, NULL); =20 @@ -126,7 +211,13 @@ void vhost_shadow_vq_stop(struct vhost_dev *dev, error_report("Couldn't restore vq kick fd: %s", strerror(-r)); } =20 + assert(!dev->shadow_vqs_enabled); + event_notifier_set_handler(&svq->host_notifier, NULL); + + /* Restore vhost call */ + vhost_virtqueue_mask(dev, dev->vdev, dev->vq_index + idx, + dev->vqs[idx].notifier_is_masked); } =20 /* @@ -154,6 +245,9 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_= dev *dev, int idx) } =20 svq->vq =3D virtio_get_queue(dev->vdev, vq_idx); + svq->vdev =3D dev->vdev; + event_notifier_set_handler(&svq->call_notifier, + vhost_shadow_vq_handle_call); return g_steal_pointer(&svq); =20 err_init_call_notifier: @@ -169,6 +263,7 @@ err_init_kick_notifier: void vhost_shadow_vq_free(VhostShadowVirtqueue *vq) { event_notifier_cleanup(&vq->kick_notifier); + event_notifier_set_handler(&vq->call_notifier, NULL); event_notifier_cleanup(&vq->call_notifier); g_free(vq); } diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 84091b5251..9c9c63345b 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1591,6 +1591,21 @@ void vhost_virtqueue_mask(struct vhost_dev *hdev, Vi= rtIODevice *vdev, int n, /* should only be called after backend is connected */ assert(hdev->vhost_ops); =20 + if (hdev->shadow_vqs_enabled) { + if (mask) { + vhost_shadow_vq_mask(hdev->shadow_vqs[index], + &hdev->vqs[index].masked_notifier); + } else { + vhost_shadow_vq_unmask(hdev->shadow_vqs[index]); + } + + /* + * Vhost call fd must remain the same since shadow vq is not polli= ng + * for changes + */ + return; + } + if (mask) { assert(vdev->use_guest_notifier_mask); file.fd =3D event_notifier_get_fd(&hdev->vqs[index].masked_notifie= r); --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442341; cv=none; d=zohomail.com; s=zohoarc; b=AniZ1/Y7qe4oR/W0+0LIr0HYznKMlyyUFDVQRVqY1nr/grmHk7sD5gttt8a3S75Tnksk0vanxl4qA5kZrRydMjga1DqngE+47NOk/z31qIY//Fo81YzTD1w3g0U3ZlNcAFJVinVbLuAiQhpnB6mFunRDJG6yg2lymIUoDqK2Rwo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442341; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=cBra4ZJJZMDqGQDXeQJipcotZaMkdrf847TS/vCHij4=; b=QE6G3maRx0rgO3njNXHunQi1CjwxPoU0ckkklHijPFfYSXQ76yCNzwqJRZu+EO+wRT3/UpC6wnEDyQspLf77sNR322DICQDfoKKWyT1Y+nlhwtRXnZUz+DqvJ9guDMgqUQv63qouACEyL8ubU6o/beRH/AFfPjWx6/MijaWaxPY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442341409663.9251517118166; Wed, 19 May 2021 09:39:01 -0700 (PDT) Received: from localhost ([::1]:45174 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPCq-000828-C6 for importer@patchew.org; Wed, 19 May 2021 12:38:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34676) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP5f-0003j9-KR for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:56819) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP5d-0005W1-Tw for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:35 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-270-m1Yd4A-3NjOhezsZDz7ahw-1; Wed, 19 May 2021 12:30:31 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 690621007477; Wed, 19 May 2021 16:30:30 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id D75DA5D6AC; Wed, 19 May 2021 16:30:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441833; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cBra4ZJJZMDqGQDXeQJipcotZaMkdrf847TS/vCHij4=; b=RFRsrTir/XUFDj45F7QklKN06TavUPoTJrwiTgVBxH32Bvaw7OfgeKTI3dxZkL9eHknXAC wqJ6MSvPywcLJEFrm/KtMA04lTKe8Ca/2Rkd924ylI6j+UAiYLViQYvjqbmnfU/xIx85yr LY4czL+DwF4UOPrGGSjNz+el0JuI1tQ= X-MC-Unique: m1Yd4A-3NjOhezsZDz7ahw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 09/29] vhost: Avoid re-set masked notifier in shadow vq Date: Wed, 19 May 2021 18:28:43 +0200 Message-Id: <20210519162903.1172366-10-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Since all the shadow virtqueue device is done in software, we can avoid the write syscall. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 7d76e271a5..c22acb4605 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -35,6 +35,9 @@ typedef struct VhostShadowVirtqueue { /* (Possible) masked notifier */ struct { EventNotifier *n; + + /* Avoid re-sending signals */ + bool signaled; } masked_notifier; =20 /* Virtio queue shadowing */ @@ -70,7 +73,8 @@ static void vhost_shadow_vq_handle_call_no_test(EventNoti= fier *n) unsigned n =3D virtio_get_queue_index(svq->vq); virtio_queue_invalidate_signalled_used(svq->vdev, n); virtio_notify_irqfd(svq->vdev, svq->vq); - } else { + } else if (!svq->masked_notifier.signaled) { + svq->masked_notifier.signaled =3D true; event_notifier_set(svq->masked_notifier.n); } } @@ -93,6 +97,7 @@ static void vhost_shadow_vq_handle_call(EventNotifier *n) */ void vhost_shadow_vq_mask(VhostShadowVirtqueue *svq, EventNotifier *masked) { + svq->masked_notifier.signaled =3D false; svq->masked_notifier.n =3D masked; } =20 --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442280; cv=none; d=zohomail.com; s=zohoarc; b=X8vFJvVi6QUUV3njGXUeYqGqvpw0yw2NeCDfe9oqzMKR+xZJBa/W0YGfq3LytdDmEh+KJ17G+yXSRFjoB3+pzmdzkDvYI1ybaLptOyTJ1wOsMyhe8gn1LBFEhOA/bxX80wqNPPPxKU+8mmZoSwX0N22brt8+s3ZnMmiDTpVYBuY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442280; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=6apyqZZ19wN+l2UaInpfg/+01UilPFGVwxMDnB8RP54=; b=G7YC0c80E/dFQ8mR79EiWn/L11bU8brKkmCXuR3ac34PB6Zi8IG/n98M/oAcR004CkvNrKcfkhrjPNSZhyIXS4KNg5D+TZcKtH1N45zLZBIKF4cH1nIFA4KFV/VDzpsw+5tJIGCeICSQgeS+Ni4rVlVwzsWRf/vHFMbzn8+Nh58= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442280148901.8908903059007; Wed, 19 May 2021 09:38:00 -0700 (PDT) Received: from localhost ([::1]:45004 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPCo-0007uB-Ms for importer@patchew.org; Wed, 19 May 2021 12:37:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34700) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP5q-0003tC-0p for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47210) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP5l-0005YI-AQ for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:45 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-263-QrbVj6CoO8uKxg8WchdkPw-1; Wed, 19 May 2021 12:30:37 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 92257801107; Wed, 19 May 2021 16:30:35 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id DE8555D6AC; Wed, 19 May 2021 16:30:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6apyqZZ19wN+l2UaInpfg/+01UilPFGVwxMDnB8RP54=; b=aFTdLtLOXpZlXJbZ/6nfmzcrqA24r1IwSpFtWTLU4Q6EjbuODpp6JhW/Zqfjwg/ut92NU6 wwqiro8vFfUIMWP6ILBU3eesodZ4LcmnudQSASoFWAbDDQYfTSIFRPWxAKRciv4tOuka66 MvkAc8HpULM3eehecvhOPojk9YHv9aI= X-MC-Unique: QrbVj6CoO8uKxg8WchdkPw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 10/29] virtio: Add vhost_shadow_vq_get_vring_addr Date: Wed, 19 May 2021 18:28:44 +0200 Message-Id: <20210519162903.1172366-11-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) It reports the shadow virtqueue address from qemu virtual address space Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 4 +++ hw/virtio/vhost-shadow-virtqueue.c | 46 ++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 2ca4b92b12..725091bc97 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -19,6 +19,10 @@ typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 void vhost_shadow_vq_mask(VhostShadowVirtqueue *svq, EventNotifier *masked= ); void vhost_shadow_vq_unmask(VhostShadowVirtqueue *svq); +void vhost_shadow_vq_get_vring_addr(const VhostShadowVirtqueue *svq, + struct vhost_vring_addr *addr); +size_t vhost_shadow_vq_driver_area_size(const VhostShadowVirtqueue *svq); +size_t vhost_shadow_vq_device_area_size(const VhostShadowVirtqueue *svq); =20 bool vhost_shadow_vq_start(struct vhost_dev *dev, unsigned idx, diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index c22acb4605..ff50f12410 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -17,6 +17,9 @@ =20 /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { + /* Shadow vring */ + struct vring vring; + /* Shadow kick notifier, sent to vhost */ EventNotifier kick_notifier; /* Shadow call notifier, sent to vhost */ @@ -114,6 +117,35 @@ void vhost_shadow_vq_unmask(VhostShadowVirtqueue *svq) svq->masked_notifier.n =3D NULL; } =20 +/* + * Get the shadow vq vring address. + * @svq Shadow virtqueue + * @addr Destination to store address + */ +void vhost_shadow_vq_get_vring_addr(const VhostShadowVirtqueue *svq, + struct vhost_vring_addr *addr) +{ + addr->desc_user_addr =3D (uint64_t)svq->vring.desc; + addr->avail_user_addr =3D (uint64_t)svq->vring.avail; + addr->used_user_addr =3D (uint64_t)svq->vring.used; +} + +size_t vhost_shadow_vq_driver_area_size(const VhostShadowVirtqueue *svq) +{ + uint16_t vq_idx =3D virtio_get_queue_index(svq->vq); + size_t desc_size =3D virtio_queue_get_desc_size(svq->vdev, vq_idx); + size_t avail_size =3D virtio_queue_get_avail_size(svq->vdev, vq_idx); + + return ROUND_UP(desc_size + avail_size, qemu_real_host_page_size); +} + +size_t vhost_shadow_vq_device_area_size(const VhostShadowVirtqueue *svq) +{ + uint16_t vq_idx =3D virtio_get_queue_index(svq->vq); + size_t used_size =3D virtio_queue_get_used_size(svq->vdev, vq_idx); + return ROUND_UP(used_size, qemu_real_host_page_size); +} + /* * Restore the vhost guest to host notifier, i.e., disables svq effect. */ @@ -232,6 +264,10 @@ void vhost_shadow_vq_stop(struct vhost_dev *dev, VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_dev *dev, int idx) { int vq_idx =3D dev->vq_index + idx; + unsigned num =3D virtio_queue_get_num(dev->vdev, vq_idx); + size_t desc_size =3D virtio_queue_get_desc_size(dev->vdev, vq_idx); + size_t driver_size; + size_t device_size; g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); int r; =20 @@ -251,6 +287,14 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost= _dev *dev, int idx) =20 svq->vq =3D virtio_get_queue(dev->vdev, vq_idx); svq->vdev =3D dev->vdev; + driver_size =3D vhost_shadow_vq_driver_area_size(svq); + device_size =3D vhost_shadow_vq_device_area_size(svq); + svq->vring.num =3D num; + svq->vring.desc =3D qemu_memalign(qemu_real_host_page_size, driver_siz= e); + svq->vring.avail =3D (void *)((char *)svq->vring.desc + desc_size); + memset(svq->vring.desc, 0, driver_size); + svq->vring.used =3D qemu_memalign(qemu_real_host_page_size, device_siz= e); + memset(svq->vring.used, 0, device_size); event_notifier_set_handler(&svq->call_notifier, vhost_shadow_vq_handle_call); return g_steal_pointer(&svq); @@ -270,5 +314,7 @@ void vhost_shadow_vq_free(VhostShadowVirtqueue *vq) event_notifier_cleanup(&vq->kick_notifier); event_notifier_set_handler(&vq->call_notifier, NULL); event_notifier_cleanup(&vq->call_notifier); + qemu_vfree(vq->vring.desc); + qemu_vfree(vq->vring.used); g_free(vq); } --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442565; cv=none; d=zohomail.com; s=zohoarc; b=XTPDT/v+cQ6q3gOef3Bz1COpBHyDXFzcHGakWjB3tjDinRV6n+G6DcEQ2P9n4+vwBGH8FZl2ts1q6FiaEgeg+wXZlfFsu6J5+57IjAWeeTg49AHSJJ4DPB3KAu5/IEsj06NhUJ8cen1LNWmTAu8o6cr2BRP96fV+KmmREGR9XC0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442565; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=BPDJ6rlrGlC64b964g2dBDdzErcAu+xMc+j3FyHfLC8=; b=VQ+R7l60eWWE5hJHaksVrszU5DwSp6Vm9iJYxrabe6JK+WO0Iarir4NU3HTuhbXoBp5yMCBMXDrYN4pAcK8sPlhmm/YSb6Nv3o/FmzvqjK8FLaF8H9HHhSckMWjmRSrh6oATUawX1IJhVL4YpYNd8CWfM+Xh4MbqrolJ0WA2thk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442565366597.281606695231; Wed, 19 May 2021 09:42:45 -0700 (PDT) Received: from localhost ([::1]:54384 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPHQ-0005rg-Fq for importer@patchew.org; Wed, 19 May 2021 12:42:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34768) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP63-0004EI-MG for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:57267) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP62-0005dd-4h for qemu-devel@nongnu.org; Wed, 19 May 2021 12:30:59 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-572-RJCtkZlqP4uGb6QFxJk5ow-1; Wed, 19 May 2021 12:30:53 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0BC158015F5; Wed, 19 May 2021 16:30:52 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id E87185D6AC; Wed, 19 May 2021 16:30:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441857; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BPDJ6rlrGlC64b964g2dBDdzErcAu+xMc+j3FyHfLC8=; b=Q1WbOWkPPJ9YP6Dzyi3oHSd+x6FFv0wLUKfvEMbT9zYZDESn/oXHFavkdxoACMlIcznAby FVw5Hrd0b2uQvIj9v+35nHiCakUnc9YwdI2fWry6vtGzpTlKcWQ/bwKLhEbTlp+7obNtTN dDIB8BKD6DETZYVH+TioFmiCZPTqgzA= X-MC-Unique: RJCtkZlqP4uGb6QFxJk5ow-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 11/29] vhost: Add vhost_vring_pause operation Date: Wed, 19 May 2021 18:28:45 +0200 Message-Id: <20210519162903.1172366-12-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) With this operation a device can be paused by a backend, allowing the later to ask status without the risk of the device override it. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-backend.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-ba= ckend.h index 8a6f8e2a7a..94d3323905 100644 --- a/include/hw/virtio/vhost-backend.h +++ b/include/hw/virtio/vhost-backend.h @@ -125,6 +125,8 @@ typedef int (*vhost_get_device_id_op)(struct vhost_dev = *dev, uint32_t *dev_id); =20 typedef bool (*vhost_force_iommu_op)(struct vhost_dev *dev); =20 +typedef int (*vhost_vring_pause_op)(struct vhost_dev *dev); + typedef struct VhostOps { VhostBackendType backend_type; vhost_backend_init vhost_backend_init; @@ -169,6 +171,7 @@ typedef struct VhostOps { vhost_dev_start_op vhost_dev_start; vhost_vq_get_addr_op vhost_vq_get_addr; vhost_get_device_id_op vhost_get_device_id; + vhost_vring_pause_op vhost_vring_pause; vhost_force_iommu_op vhost_force_iommu; } VhostOps; =20 --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442559; cv=none; d=zohomail.com; s=zohoarc; b=YUYflPI5KoiI/3O8jspcUs/MdQnW6uf075pv1HUjXchy3xH/+r0KsVU6g8MnxU5SU3LOjPXBjYZaiBTT7pyrjKvlmbOn31ltktlOOpK3qRLwZEfeT4rrbdAcC38qs5p/iyo01Uqw10rR7CoplpLmMgaSBA9xZJ3AjZXouu8w38U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442559; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=jpVWSZqTF2zL6nviKEiZCIGpvvk/+5NdTzfZHJLDvZQ=; b=frg/Ee/9KEfxVKVKH/26YloYcTlZ9tgoQolk4AJiEUdBZU8SHeGS3vTOlC7L82FjQpyGViXzN8gRVHSzcSIRonGbWaIxMxEynONnWv4KmKmva3W5QwiLQVHZs5dM9UaNmbJ/Ftw/1gIXzvRTLInm/5TkAPTGJqVlYfvxicqR37w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442559457167.06796906462728; Wed, 19 May 2021 09:42:39 -0700 (PDT) Received: from localhost ([::1]:53662 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPHJ-0005Nb-OJ for importer@patchew.org; Wed, 19 May 2021 12:42:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34790) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP65-0004L8-D4 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49750) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP63-0005eT-HT for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:01 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-121-D8nMnuc4MFGOubFuMZ2Ljw-1; Wed, 19 May 2021 12:30:57 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D4ED5100747E; Wed, 19 May 2021 16:30:55 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 624745D720; Wed, 19 May 2021 16:30:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441859; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jpVWSZqTF2zL6nviKEiZCIGpvvk/+5NdTzfZHJLDvZQ=; b=etorMLhgKOEQjiUe5n8dSpdkJVkm2t8Q42xO+HeCCWyqXnhoG8Am8L1VH5sEIK+xSSha6F d6s/Ku3Go6PRPo3cEbmTA0Vk6FJfOVR4WBAyDyCNP4iqkc5NypoPrsRyF9TAlc89T3faYl EoaxbTqR/tk+wst23qbIVYTuaCo99LE= X-MC-Unique: D8nMnuc4MFGOubFuMZ2Ljw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 12/29] vhost: add vhost_kernel_vring_pause Date: Wed, 19 May 2021 18:28:46 +0200 Message-Id: <20210519162903.1172366-13-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This is just a commit to allow the testing with vhost-net, not intended for the final version or any other device. vhost_kernel_vring_pause stops the device, so qemu can ask for its status (next available idx the device was going to consume) and to replace vring addresses. When SVQ starts it can resume consuming the guest's driver ring, without notice from the latter. Not stopping the device before of the swapping could imply that it process more buffers than reported, what would duplicate the device action. Mimic vhost-vdpa behavior, vhost_kernel_start is intended to resume the device. In the former it performs a full reset. Since this is a temporary commit to allow testing with vhost-net, the latter just set a new backend, that is enough for vhost-net to realize the new vring addresses. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-backend.c | 42 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c index 31b33bde37..9653b7fddb 100644 --- a/hw/virtio/vhost-backend.c +++ b/hw/virtio/vhost-backend.c @@ -201,6 +201,46 @@ static int vhost_kernel_get_vq_index(struct vhost_dev = *dev, int idx) return idx - dev->vq_index; } =20 +static int vhost_kernel_set_vq_pause(struct vhost_dev *dev, unsigned idx, + bool pause) +{ + struct vhost_vring_file file =3D { + .index =3D idx, + }; + + if (pause) { + file.fd =3D -1; /* Pass -1 to unbind from file. */ + } else { + struct vhost_net *vn_dev =3D container_of(dev, struct vhost_net, d= ev); + file.fd =3D vn_dev->backend; + } + + return vhost_kernel_net_set_backend(dev, &file); +} + +static int vhost_kernel_vring_pause(struct vhost_dev *dev) +{ + int i; + + for (i =3D 0; i < dev->nvqs; ++i) { + vhost_kernel_set_vq_pause(dev, i, true); + } + + return 0; +} + +static int vhost_kernel_start(struct vhost_dev *dev, bool start) +{ + int i; + + assert(start); + for (i =3D 0; i < dev->nvqs; ++i) { + vhost_kernel_set_vq_pause(dev, i, false); + } + + return 0; +} + #ifdef CONFIG_VHOST_VSOCK static int vhost_kernel_vsock_set_guest_cid(struct vhost_dev *dev, uint64_t guest_cid) @@ -317,6 +357,8 @@ static const VhostOps kernel_ops =3D { .vhost_set_owner =3D vhost_kernel_set_owner, .vhost_reset_device =3D vhost_kernel_reset_device, .vhost_get_vq_index =3D vhost_kernel_get_vq_index, + .vhost_dev_start =3D vhost_kernel_start, + .vhost_vring_pause =3D vhost_kernel_vring_pause, #ifdef CONFIG_VHOST_VSOCK .vhost_vsock_set_guest_cid =3D vhost_kernel_vsock_set_guest_cid, .vhost_vsock_set_running =3D vhost_kernel_vsock_set_running, --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442829; cv=none; d=zohomail.com; s=zohoarc; b=S3RE8FJuvphwnPC+0FrRI28Smlj8TrWIIOBBPFOjgGyb3g0HmEC/qIeKPGOR/1Sld2qadTW24CeVt4yWwGchuF+JYasCjWb1lzI9bziDUjIw02ZoWV+UJFo7JB3q04bOhREKXtPaxGreuePqaeuyTp49ick38t0og27uP365HhA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442829; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=L2lLIl9UsgREXs2nPJJjVHAoVYw+HdUzi/kUksrz3ms=; b=lDn6k2dbIZPB0Wiu2uTzQ/bj+lyBu4sa/ZJFh/j8DMzHovV/0y2s/2ez/L8rtL4bjzRm7v0b4cJ0DimJWVn67lMw3YYZURZfv5JAm4WaBNpFI1U1jEhKcQ6eGBr0vv0j4edvPzjNRpRclIgL9IcH0QQgXyrwGl30iONS4OLJ2Us= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442829594524.7211150637407; Wed, 19 May 2021 09:47:09 -0700 (PDT) Received: from localhost ([::1]:34926 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPLg-0003Zi-M7 for importer@patchew.org; Wed, 19 May 2021 12:47:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34838) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP69-0004ZC-S6 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:05 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:58322) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP67-0005fl-OK for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:05 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-447-l-anlM_ZMQKjN4oyPi9KlA-1; Wed, 19 May 2021 12:31:00 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2EA80180FD69; Wed, 19 May 2021 16:30:59 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3DC5F5D6AC; Wed, 19 May 2021 16:30:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441863; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L2lLIl9UsgREXs2nPJJjVHAoVYw+HdUzi/kUksrz3ms=; b=SQB9dl1VT+jxN/gBdHd4xofx+1fC3gIkuwPAsXLOyTh/nQ3drE7EeU6VLDq6InqciPcK4T Z+GT38dygA2o2NYC34LkdBq4B1qpfecO6m5L7ObKBN0/x/yVZ1WqjWVytvGfAuF8Zc6dAb kongxf4k4QqtewzFy+5qnnGZ3rQwUT4= X-MC-Unique: l-anlM_ZMQKjN4oyPi9KlA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 13/29] vhost: Add vhost_get_iova_range operation Date: Wed, 19 May 2021 18:28:47 +0200 Message-Id: <20210519162903.1172366-14-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) For simplicity, If a device does not support this operation it means that it can handle full (uint64_t)-1 iova address. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-backend.h | 5 +++++ hw/virtio/vhost-vdpa.c | 18 ++++++++++++++++++ hw/virtio/trace-events | 1 + 3 files changed, 24 insertions(+) diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-ba= ckend.h index 94d3323905..bcb112c166 100644 --- a/include/hw/virtio/vhost-backend.h +++ b/include/hw/virtio/vhost-backend.h @@ -36,6 +36,7 @@ struct vhost_vring_addr; struct vhost_scsi_target; struct vhost_iotlb_msg; struct vhost_virtqueue; +struct vhost_vdpa_iova_range; =20 typedef int (*vhost_backend_init)(struct vhost_dev *dev, void *opaque); typedef int (*vhost_backend_cleanup)(struct vhost_dev *dev); @@ -127,6 +128,9 @@ typedef bool (*vhost_force_iommu_op)(struct vhost_dev *= dev); =20 typedef int (*vhost_vring_pause_op)(struct vhost_dev *dev); =20 +typedef int (*vhost_get_iova_range)(struct vhost_dev *dev, + hwaddr *first, hwaddr *last); + typedef struct VhostOps { VhostBackendType backend_type; vhost_backend_init vhost_backend_init; @@ -173,6 +177,7 @@ typedef struct VhostOps { vhost_get_device_id_op vhost_get_device_id; vhost_vring_pause_op vhost_vring_pause; vhost_force_iommu_op vhost_force_iommu; + vhost_get_iova_range vhost_get_iova_range; } VhostOps; =20 extern const VhostOps user_ops; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 01d2101d09..74fe92935e 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -579,6 +579,23 @@ static bool vhost_vdpa_force_iommu(struct vhost_dev *= dev) return true; } =20 +static int vhost_vdpa_get_iova_range(struct vhost_dev *dev, + hwaddr *first, hwaddr *last) +{ + int ret; + struct vhost_vdpa_iova_range range; + + ret =3D vhost_vdpa_call(dev, VHOST_VDPA_GET_IOVA_RANGE, &range); + if (ret !=3D 0) { + return ret; + } + + *first =3D range.first; + *last =3D range.last; + trace_vhost_vdpa_get_iova_range(dev, *first, *last); + return ret; +} + const VhostOps vdpa_ops =3D { .backend_type =3D VHOST_BACKEND_TYPE_VDPA, .vhost_backend_init =3D vhost_vdpa_init, @@ -611,4 +628,5 @@ const VhostOps vdpa_ops =3D { .vhost_get_device_id =3D vhost_vdpa_get_device_id, .vhost_vq_get_addr =3D vhost_vdpa_vq_get_addr, .vhost_force_iommu =3D vhost_vdpa_force_iommu, + .vhost_get_iova_range =3D vhost_vdpa_get_iova_range, }; diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index c62727f879..5debe3a681 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -52,6 +52,7 @@ vhost_vdpa_set_vring_call(void *dev, unsigned int index, = int fd) "dev: %p index: vhost_vdpa_get_features(void *dev, uint64_t features) "dev: %p features: 0= x%"PRIx64 vhost_vdpa_set_owner(void *dev) "dev: %p" vhost_vdpa_vq_get_addr(void *dev, void *vq, uint64_t desc_user_addr, uint6= 4_t avail_user_addr, uint64_t used_user_addr) "dev: %p vq: %p desc_user_add= r: 0x%"PRIx64" avail_user_addr: 0x%"PRIx64" used_user_addr: 0x%"PRIx64 +vhost_vdpa_get_iova_range(void *dev, uint64_t first, uint64_t last) "dev: = %p first: 0x%"PRIx64" last: 0x%"PRIx64 =20 # virtio.c virtqueue_alloc_element(void *elem, size_t sz, unsigned in_num, unsigned o= ut_num) "elem %p size %zd in_num %u out_num %u" --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621443054; cv=none; d=zohomail.com; s=zohoarc; b=U2QVkiEwAqvNMyp6t+MZ9cF1NVsQydHAXufbtZHmAzsMRp0QioG7ftxycg+xPf74BMg4ZN+6jpqw0cQeebCCs6WmuBJ20uuVWz63tb/82uxQfNIumKXQf6sMMT1GJfvpmvBrO4xGHTO00TjK1546akQLTbBtE5/QCLddDlK96UM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621443054; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=eDk6ORb0CFQX0SjnjEy5Mjgwnx7Of5HDLZ0WqbSxD4w=; b=KV4wImSiCuDNGP5JViNNMtQG4LkX0NuJIFUg5Hr8PB1/NpQsJyb/XwmcVCoepRRQ7VeAUmfny1WA8GkFQ0IswRGLmJ97aaQc32B1pvhUXTCo/ZKlJXyvNt4GRHWpjfn7WTx7fKuS8oDUuYUvOsXzRcGTU0QCzTaXGOxtMHSnu/U= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621443054127158.71895620881185; Wed, 19 May 2021 09:50:54 -0700 (PDT) Received: from localhost ([::1]:43706 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPPI-0001FB-WF for importer@patchew.org; Wed, 19 May 2021 12:50:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34862) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6C-0004kC-N0 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:35956) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6B-0005h2-2E for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:08 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-272-OB_nGhGKP5y1vWnP045JCA-1; Wed, 19 May 2021 12:31:03 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 72BFE108BD0C; Wed, 19 May 2021 16:31:02 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 859ED5D6D5; Wed, 19 May 2021 16:30:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eDk6ORb0CFQX0SjnjEy5Mjgwnx7Of5HDLZ0WqbSxD4w=; b=c9qJE3yIss8CfECwZSLXDq4OEY8M2KTVbfziJFGOTR0jDne8v41O0fn/HjVLvg1LDIphYm 95WVYwm8kMqAtsIR/2LC9oaJACN8Ze9bR9ugmftCD7/7BPUBUSaw4WdpPmAkNAAzU0omGX pxCaFJPd01B0F9crQ0Jx7WV6gFWuEAY= X-MC-Unique: OB_nGhGKP5y1vWnP045JCA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 14/29] vhost: add vhost_has_limited_iova_range Date: Wed, 19 May 2021 18:28:48 +0200 Message-Id: <20210519162903.1172366-15-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost.h | 5 +++++ hw/virtio/vhost.c | 17 +++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 67cedf83da..c97a4c0017 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -88,6 +88,10 @@ struct vhost_dev { bool log_enabled; bool shadow_vqs_enabled; uint64_t log_size; + struct { + hwaddr first; + hwaddr last; + } iova_range; VhostShadowVirtqueue **shadow_vqs; Error *migration_blocker; const VhostOps *vhost_ops; @@ -129,6 +133,7 @@ uint64_t vhost_get_features(struct vhost_dev *hdev, con= st int *feature_bits, void vhost_ack_features(struct vhost_dev *hdev, const int *feature_bits, uint64_t features); bool vhost_has_free_slot(void); +bool vhost_has_limited_iova_range(const struct vhost_dev *hdev); =20 int vhost_net_set_backend(struct vhost_dev *hdev, struct vhost_vring_file *file); diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 9c9c63345b..333877ca3b 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1386,6 +1386,18 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opa= que, goto fail; } =20 + if (hdev->vhost_ops->vhost_get_iova_range) { + r =3D hdev->vhost_ops->vhost_get_iova_range(hdev, + &hdev->iova_range.first, + &hdev->iova_range.last); + if (unlikely(r !=3D 0)) { + error_report("Can't request IOVA range"); + goto fail; + } + } else { + hdev->iova_range.last =3D (hwaddr)-1; + } + for (i =3D 0; i < hdev->nvqs; ++i, ++n_initialized_vqs) { r =3D vhost_virtqueue_init(hdev, hdev->vqs + i, hdev->vq_index + i= ); if (r < 0) { @@ -1622,6 +1634,11 @@ void vhost_virtqueue_mask(struct vhost_dev *hdev, Vi= rtIODevice *vdev, int n, } } =20 +bool vhost_has_limited_iova_range(const struct vhost_dev *hdev) +{ + return hdev->iova_range.first || hdev->iova_range.last !=3D HWADDR_MAX; +} + uint64_t vhost_get_features(struct vhost_dev *hdev, const int *feature_bit= s, uint64_t features) { --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442696; cv=none; d=zohomail.com; s=zohoarc; b=iL+bHOqkfX6ON6V7XevjjmOCGY/AOnGye9vDgZJ+x+/wlM0NG1tde7IkEM6W4I9jdU3mwtbvOdNmhf5qhL1WFP1TZMX27UcVTBTRYrlJjsJ2Lcd8VXIhkXIeQtZLlPjIN874KmcIbRjNvSXFvN5xWenfT4xMMpauXSUalX1SbZ4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442696; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=wr2UYo9RlDyRTsteJfGPVne0NzOjWnR9hLHKjUWdu9c=; b=CblJSROk6MkXpfT9qujLHnOrHoNUP1p706WJ6F0esn0xgM//UkNfIaAaaMU64nIa8+HSCzAqL+a68UxwX+vpXiGEx0oIttD1uymsvSvAJztMZWDZmjQE3RS201AsqLnvpv2pa2xpBX2u+ETip6cNyNVUAxsR9OUs7zQIfyLvRsI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442696449991.3383496064716; Wed, 19 May 2021 09:44:56 -0700 (PDT) Received: from localhost ([::1]:57826 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPJW-0008FM-69 for importer@patchew.org; Wed, 19 May 2021 12:44:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34990) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6V-0005FB-OP for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:53074) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6U-0005qf-0C for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-249-x8UrcRI7N8Su0WF6wc_DjQ-1; Wed, 19 May 2021 12:31:23 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 86BE8107ACCD; Wed, 19 May 2021 16:31:21 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id C9A765D6AC; Wed, 19 May 2021 16:31:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441885; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wr2UYo9RlDyRTsteJfGPVne0NzOjWnR9hLHKjUWdu9c=; b=Koe4E6A2IsuUgQtZarcJbJe9U00xcZsO9szr35lSKcpFsv+NRSgUME1i377KOn1b7bmUXm U06hmjTJfpspZWCMxhoGancdXf7O3oOIBQkpuKn1pF+gc/ER4tOxGlKHlsuaU0knkJdDHc AspTtbv8UiF2CdYujm709xoC/MV/K2E= X-MC-Unique: x8UrcRI7N8Su0WF6wc_DjQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 15/29] vhost: Add enable_custom_iommu to VhostOps Date: Wed, 19 May 2021 18:28:49 +0200 Message-Id: <20210519162903.1172366-16-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This operation enable the backend-specific IOTLB entries. If a backend support this, it start managing its own entries, and vhost can disable it through this operation and recover control. Every enable/disable operation must also clear all IOTLB device entries. At the moment, the only backend that does so is vhost-vdpa. To fully support these, vdpa needs also to expose a way for vhost subsystem to map and unmap entries. This will be done in future commits. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-backend.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-ba= ckend.h index bcb112c166..f8eed2ace5 100644 --- a/include/hw/virtio/vhost-backend.h +++ b/include/hw/virtio/vhost-backend.h @@ -128,6 +128,9 @@ typedef bool (*vhost_force_iommu_op)(struct vhost_dev *= dev); =20 typedef int (*vhost_vring_pause_op)(struct vhost_dev *dev); =20 +typedef int (*vhost_enable_custom_iommu_op)(struct vhost_dev *dev, + bool enable); + typedef int (*vhost_get_iova_range)(struct vhost_dev *dev, hwaddr *first, hwaddr *last); =20 @@ -177,6 +180,7 @@ typedef struct VhostOps { vhost_get_device_id_op vhost_get_device_id; vhost_vring_pause_op vhost_vring_pause; vhost_force_iommu_op vhost_force_iommu; + vhost_enable_custom_iommu_op vhost_enable_custom_iommu; vhost_get_iova_range vhost_get_iova_range; } VhostOps; =20 --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442296; cv=none; d=zohomail.com; s=zohoarc; b=QYANEiAO3h8YgFdd3mpkdSZ1s7+1pO/9k9SB2TJLaH+3zLa15uSzeTJJg+7mfJhXIUYpkSWfr9GV4s4YQDSbNvw30aj42tXlsHsnEwOKHVPk0js10FTZazO+MhY0nx6FhwY1apmZJyb305O0OoZcc7Lib5cm0lB1ZepO9pfox+I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442296; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=2uLkutZRWMTAoyzRaj5p0L8+Fz0EO+Zt2kvFYEu9WJc=; b=YMo6E7hjknW2CGLk+EgLARyooifwkRuiDmobghgex4NSPCxhamxqAFSW9mi0aEDhrFlHthT5JWPMwoH7ECIJveH6O39CAnz0z5E8NfD7qz55O8XHR5X2D2FJY84lzet2l+IbmBS4ql6UmZ7uHEe5tYBzsI0DAGUvJfEIsZFmZ3s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 162144229669644.524046259688475; Wed, 19 May 2021 09:38:16 -0700 (PDT) Received: from localhost ([::1]:46326 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPD1-0000Ku-8v for importer@patchew.org; Wed, 19 May 2021 12:38:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35016) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6Z-0005Sk-7n for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:31492) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6X-0005tL-Ds for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:30 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-204-Ll5Pzg1bPWek6Db17Ar7Hw-1; Wed, 19 May 2021 12:31:26 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D09D3180FD6E; Wed, 19 May 2021 16:31:24 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id DB4695D6AC; Wed, 19 May 2021 16:31:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441888; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2uLkutZRWMTAoyzRaj5p0L8+Fz0EO+Zt2kvFYEu9WJc=; b=cSLuIdwrHhJgDVXvIdy/UTXkvmzD64l4BxTttKpurmQN1iM6RWwjNzAFzid77M7dqBK0HE f1nefFipIzaHW3zApYWnLj5W3TSiAnluG2LqpIMrNIQcwC7nj4wDEShLm3kGTl1E/SFI7U 1vtWWoW2kx4M+v8eF0QAIBmvzc6DoEY= X-MC-Unique: Ll5Pzg1bPWek6Db17Ar7Hw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 16/29] vhost-vdpa: Add vhost_vdpa_enable_custom_iommu Date: Wed, 19 May 2021 18:28:50 +0200 Message-Id: <20210519162903.1172366-17-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Implementation of vhost_ops->enable_custom_iommu Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-vdpa.c | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 74fe92935e..9e7a0ce5e0 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -272,6 +272,29 @@ static void vhost_vdpa_add_status(struct vhost_dev *de= v, uint8_t status) vhost_vdpa_call(dev, VHOST_VDPA_SET_STATUS, &s); } =20 +static int vhost_vdpa_enable_custom_iommu(struct vhost_dev *dev, bool enab= le) +{ + struct vhost_vdpa *v =3D dev->opaque; + hwaddr iova_range_last =3D dev->iova_range.last; + if (iova_range_last !=3D (hwaddr)-1) { + iova_range_last++; + } + + if (enable) { + int r =3D vhost_vdpa_dma_unmap(v, dev->iova_range.first, iova_rang= e_last); + if (r !=3D 0) { + error_report("Fail to invalidate device iotlb"); + } + + memory_listener_register(&v->listener, &address_space_memory); + } else { + memory_listener_unregister(&v->listener); + return vhost_vdpa_dma_unmap(v, dev->iova_range.first, iova_range_l= ast); + } + + return 0; +} + static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque) { struct vhost_vdpa *v; @@ -299,7 +322,7 @@ static int vhost_vdpa_cleanup(struct vhost_dev *dev) assert(dev->vhost_ops->backend_type =3D=3D VHOST_BACKEND_TYPE_VDPA); v =3D dev->opaque; trace_vhost_vdpa_cleanup(dev, v); - memory_listener_unregister(&v->listener); + vhost_vdpa_enable_custom_iommu(dev, false); =20 dev->opaque =3D NULL; return 0; @@ -470,11 +493,10 @@ static int vhost_vdpa_get_config(struct vhost_dev *de= v, uint8_t *config, =20 static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) { - struct vhost_vdpa *v =3D dev->opaque; trace_vhost_vdpa_dev_start(dev, started); if (started) { uint8_t status =3D 0; - memory_listener_register(&v->listener, &address_space_memory); + vhost_vdpa_enable_custom_iommu(dev, true); vhost_vdpa_set_vring_ready(dev); vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK); vhost_vdpa_call(dev, VHOST_VDPA_GET_STATUS, &status); @@ -484,7 +506,7 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev, = bool started) vhost_vdpa_reset_device(dev); vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER); - memory_listener_unregister(&v->listener); + vhost_vdpa_enable_custom_iommu(dev, false); =20 return 0; } @@ -628,5 +650,6 @@ const VhostOps vdpa_ops =3D { .vhost_get_device_id =3D vhost_vdpa_get_device_id, .vhost_vq_get_addr =3D vhost_vdpa_vq_get_addr, .vhost_force_iommu =3D vhost_vdpa_force_iommu, + .vhost_enable_custom_iommu =3D vhost_vdpa_enable_custom_iommu, .vhost_get_iova_range =3D vhost_vdpa_get_iova_range, }; --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621443209; cv=none; d=zohomail.com; s=zohoarc; b=LjKOcc4CaMzFjL8L+5ldqTldYDMmIFQS5Y5tZzK04UTa/F+GhgZ/QYYwjL31roLlb9xwI7thy2DRQXiXj/OFJdmSGfH8x6pGGjImBniLKU4reg1H1R+slQW12ubgEF2mMXnxkmgHn8GvU73CQgJm7jQ4wsnmQVdJPufdvx0y0xY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621443209; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=eNwtx5EEHBP81WChXzxc5VnAMW2cP+sOPItLL048mGw=; b=HbYng9OWwyO0bHuhmSbw3HjTVrjFjxu/IXJFUzu2ZcID71JsKmNK9orDo8o+cPpwqOgU62yvaSL/wvRXOX7EXoTbqxZOW//i+OjKrorCKzfxzsfyuv3Vv8Z6vHcqfWXIt+iysgxAYZ0+0Hqv7sq6QvSCEZDhe1JpRtR2dyzasLs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621443209560510.4502465627753; Wed, 19 May 2021 09:53:29 -0700 (PDT) Received: from localhost ([::1]:52328 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPRo-00079A-Pz for importer@patchew.org; Wed, 19 May 2021 12:53:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35092) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6g-0005u6-De for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:58794) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6d-0005vN-5n for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:38 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-601--gC4mgEBPsKU5tR2yJzU5g-1; Wed, 19 May 2021 12:31:30 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4E7E3801817; Wed, 19 May 2021 16:31:28 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 310BE5D6AC; Wed, 19 May 2021 16:31:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441894; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eNwtx5EEHBP81WChXzxc5VnAMW2cP+sOPItLL048mGw=; b=fgLJ+r0kG/OBaH+rpW6D7q6Le7dzLyy7/XOu7NOZ0OFlCSEGVuuhR4Azji6SADwKDC0AJV qADCwsakNV/s+6aPeIp97la7eGlSscIgwFi9t+khaeYBIfE0GojVaz3KeSbSdt624wOv1E a+Swv/yjqVuqCeJ3NxibFyRCdHTZzi4= X-MC-Unique: -gC4mgEBPsKU5tR2yJzU5g-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 17/29] vhost: Shadow virtqueue buffers forwarding Date: Wed, 19 May 2021 18:28:51 +0200 Message-Id: <20210519162903.1172366-18-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Initial version of shadow virtqueue that actually forward buffers. The exposed addresses are the qemu's virtual address, so devices with IOMMU that does not allow full mapping of qemu's address space does not work at the moment. Also for simplicity it only supports modern devices, that expects vring in little endian, with split ring and no event idx or indirect descriptors. It reuses the VirtQueue code for the device part. The driver part is based on Linux's virtio_ring driver, but with stripped functionality and optimizations so it's easier to review. Later commits will solve some of these concerns. Code also need to map used ring (device part) as RW in, and only in, vhost-net. To map (or call vhost_device_iotlb_miss) inconditionally would print an error in case of vhost devices with its own mapping (vdpa). To know if this call is needed, vhost_sw_live_migration_start_vq and vhost_sw_live_migration_stop copy the test performed in vhost_dev_start. Testing for the actual backend type could be cleaner, or checking for non-NULL vhost_force_iommu, enable_custom_iommu, or another vhostOp. We could extract this test in its own function too, so its name could give a better hint. Just copy the vhost_dev_start check at the moment. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 205 +++++++++++++++++++++++++++-- hw/virtio/vhost.c | 134 ++++++++++++++++++- 2 files changed, 325 insertions(+), 14 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index ff50f12410..6d767fe248 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -9,6 +9,7 @@ =20 #include "hw/virtio/vhost-shadow-virtqueue.h" #include "hw/virtio/vhost.h" +#include "hw/virtio/virtio-access.h" =20 #include "standard-headers/linux/vhost_types.h" =20 @@ -48,9 +49,93 @@ typedef struct VhostShadowVirtqueue { =20 /* Virtio device */ VirtIODevice *vdev; + + /* Map for returning guest's descriptors */ + VirtQueueElement **ring_id_maps; + + /* Next head to expose to device */ + uint16_t avail_idx_shadow; + + /* Next free descriptor */ + uint16_t free_head; + + /* Last seen used idx */ + uint16_t shadow_used_idx; + + /* Next head to consume from device */ + uint16_t used_idx; } VhostShadowVirtqueue; =20 -/* Forward guest notifications */ +static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, + const struct iovec *iovec, + size_t num, bool more_descs, bool writ= e) +{ + uint16_t i =3D svq->free_head, last =3D svq->free_head; + unsigned n; + uint16_t flags =3D write ? cpu_to_le16(VRING_DESC_F_WRITE) : 0; + vring_desc_t *descs =3D svq->vring.desc; + + if (num =3D=3D 0) { + return; + } + + for (n =3D 0; n < num; n++) { + if (more_descs || (n + 1 < num)) { + descs[i].flags =3D flags | cpu_to_le16(VRING_DESC_F_NEXT); + } else { + descs[i].flags =3D flags; + } + descs[i].addr =3D cpu_to_le64((hwaddr)iovec[n].iov_base); + descs[i].len =3D cpu_to_le32(iovec[n].iov_len); + + last =3D i; + i =3D cpu_to_le16(descs[i].next); + } + + svq->free_head =3D le16_to_cpu(descs[last].next); +} + +static unsigned vhost_shadow_vq_add_split(VhostShadowVirtqueue *svq, + VirtQueueElement *elem) +{ + int head; + unsigned avail_idx; + vring_avail_t *avail =3D svq->vring.avail; + + head =3D svq->free_head; + + /* We need some descriptors here */ + assert(elem->out_num || elem->in_num); + + vhost_vring_write_descs(svq, elem->out_sg, elem->out_num, + elem->in_num > 0, false); + vhost_vring_write_descs(svq, elem->in_sg, elem->in_num, false, true); + + /* + * Put entry in available array (but don't update avail->idx until they + * do sync). + */ + avail_idx =3D svq->avail_idx_shadow & (svq->vring.num - 1); + avail->ring[avail_idx] =3D cpu_to_le16(head); + svq->avail_idx_shadow++; + + /* Expose descriptors to device */ + smp_wmb(); + avail->idx =3D cpu_to_le16(svq->avail_idx_shadow); + + return head; + +} + +static void vhost_shadow_vq_add(VhostShadowVirtqueue *svq, + VirtQueueElement *elem) +{ + unsigned qemu_head =3D vhost_shadow_vq_add_split(svq, elem); + + svq->ring_id_maps[qemu_head] =3D elem; +} + +/* Handle guest->device notifications */ static void vhost_handle_guest_kick(EventNotifier *n) { VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, @@ -60,7 +145,67 @@ static void vhost_handle_guest_kick(EventNotifier *n) return; } =20 - event_notifier_set(&svq->kick_notifier); + /* Make available as many buffers as possible */ + do { + if (virtio_queue_get_notification(svq->vq)) { + /* No more notifications until process all available */ + virtio_queue_set_notification(svq->vq, false); + } + + while (true) { + VirtQueueElement *elem =3D virtqueue_pop(svq->vq, sizeof(*elem= )); + if (!elem) { + break; + } + + vhost_shadow_vq_add(svq, elem); + event_notifier_set(&svq->kick_notifier); + } + + virtio_queue_set_notification(svq->vq, true); + } while (!virtio_queue_empty(svq->vq)); +} + +static bool vhost_shadow_vq_more_used(VhostShadowVirtqueue *svq) +{ + if (svq->used_idx !=3D svq->shadow_used_idx) { + return true; + } + + /* Get used idx must not be reordered */ + smp_rmb(); + svq->shadow_used_idx =3D cpu_to_le16(svq->vring.used->idx); + + return svq->used_idx !=3D svq->shadow_used_idx; +} + +static VirtQueueElement *vhost_shadow_vq_get_buf(VhostShadowVirtqueue *svq) +{ + vring_desc_t *descs =3D svq->vring.desc; + const vring_used_t *used =3D svq->vring.used; + vring_used_elem_t used_elem; + uint16_t last_used; + + if (!vhost_shadow_vq_more_used(svq)) { + return NULL; + } + + last_used =3D svq->used_idx & (svq->vring.num - 1); + used_elem.id =3D le32_to_cpu(used->ring[last_used].id); + used_elem.len =3D le32_to_cpu(used->ring[last_used].len); + + if (unlikely(used_elem.id >=3D svq->vring.num)) { + error_report("Device %s says index %u is available", svq->vdev->na= me, + used_elem.id); + return NULL; + } + + descs[used_elem.id].next =3D svq->free_head; + svq->free_head =3D used_elem.id; + + svq->used_idx++; + svq->ring_id_maps[used_elem.id]->len =3D used_elem.len; + return g_steal_pointer(&svq->ring_id_maps[used_elem.id]); } =20 /* Forward vhost notifications */ @@ -69,17 +214,33 @@ static void vhost_shadow_vq_handle_call_no_test(EventN= otifier *n) VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, call_notifier); EventNotifier *masked_notifier; + VirtQueue *vq =3D svq->vq; =20 masked_notifier =3D svq->masked_notifier.n; =20 - if (!masked_notifier) { - unsigned n =3D virtio_get_queue_index(svq->vq); - virtio_queue_invalidate_signalled_used(svq->vdev, n); - virtio_notify_irqfd(svq->vdev, svq->vq); - } else if (!svq->masked_notifier.signaled) { - svq->masked_notifier.signaled =3D true; - event_notifier_set(svq->masked_notifier.n); - } + /* Make as many buffers as possible used. */ + do { + unsigned i =3D 0; + + /* TODO: Use VRING_AVAIL_F_NO_INTERRUPT */ + while (true) { + g_autofree VirtQueueElement *elem =3D vhost_shadow_vq_get_buf(= svq); + if (!elem) { + break; + } + + assert(i < svq->vring.num); + virtqueue_fill(vq, elem, elem->len, i++); + } + + virtqueue_flush(vq, i); + if (!masked_notifier) { + virtio_notify_irqfd(svq->vdev, svq->vq); + } else if (!svq->masked_notifier.signaled) { + svq->masked_notifier.signaled =3D true; + event_notifier_set(svq->masked_notifier.n); + } + } while (vhost_shadow_vq_more_used(svq)); } =20 static void vhost_shadow_vq_handle_call(EventNotifier *n) @@ -243,7 +404,11 @@ void vhost_shadow_vq_stop(struct vhost_dev *dev, unsigned idx, VhostShadowVirtqueue *svq) { + int i; int r =3D vhost_shadow_vq_restore_vdev_host_notifier(dev, idx, svq); + + assert(!dev->shadow_vqs_enabled); + if (unlikely(r < 0)) { error_report("Couldn't restore vq kick fd: %s", strerror(-r)); } @@ -255,6 +420,18 @@ void vhost_shadow_vq_stop(struct vhost_dev *dev, /* Restore vhost call */ vhost_virtqueue_mask(dev, dev->vdev, dev->vq_index + idx, dev->vqs[idx].notifier_is_masked); + + + for (i =3D 0; i < svq->vring.num; ++i) { + g_autofree VirtQueueElement *elem =3D svq->ring_id_maps[i]; + /* + * Although the doc says we must unpop in order, it's ok to unpop + * everything. + */ + if (elem) { + virtqueue_unpop(svq->vq, elem, elem->len); + } + } } =20 /* @@ -269,7 +446,7 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_= dev *dev, int idx) size_t driver_size; size_t device_size; g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); - int r; + int r, i; =20 r =3D event_notifier_init(&svq->kick_notifier, 0); if (r !=3D 0) { @@ -295,6 +472,11 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost= _dev *dev, int idx) memset(svq->vring.desc, 0, driver_size); svq->vring.used =3D qemu_memalign(qemu_real_host_page_size, device_siz= e); memset(svq->vring.used, 0, device_size); + for (i =3D 0; i < num - 1; i++) { + svq->vring.desc[i].next =3D cpu_to_le16(i + 1); + } + + svq->ring_id_maps =3D g_new0(VirtQueueElement *, num); event_notifier_set_handler(&svq->call_notifier, vhost_shadow_vq_handle_call); return g_steal_pointer(&svq); @@ -314,6 +496,7 @@ void vhost_shadow_vq_free(VhostShadowVirtqueue *vq) event_notifier_cleanup(&vq->kick_notifier); event_notifier_set_handler(&vq->call_notifier, NULL); event_notifier_cleanup(&vq->call_notifier); + g_free(vq->ring_id_maps); qemu_vfree(vq->vring.desc); qemu_vfree(vq->vring.used); g_free(vq); diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 333877ca3b..5b5001a08a 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1021,6 +1021,19 @@ int vhost_device_iotlb_miss(struct vhost_dev *dev, u= int64_t iova, int write) =20 trace_vhost_iotlb_miss(dev, 1); =20 + if (dev->shadow_vqs_enabled) { + uaddr =3D iova; + len =3D 4096; + ret =3D vhost_backend_update_device_iotlb(dev, iova, uaddr, len, + IOMMU_RW); + if (ret) { + trace_vhost_iotlb_miss(dev, 2); + error_report("Fail to update device iotlb"); + } + + return ret; + } + iotlb =3D address_space_get_iotlb_entry(dev->vdev->dma_as, iova, write, MEMTXATTRS_UNSPECIFIED); @@ -1222,12 +1235,37 @@ static void vhost_virtqueue_stop(struct vhost_dev *= dev, =20 static int vhost_sw_live_migration_stop(struct vhost_dev *dev) { - int idx; + int idx, r; =20 dev->shadow_vqs_enabled =3D false; =20 + r =3D dev->vhost_ops->vhost_vring_pause(dev); + assert(r =3D=3D 0); + if (vhost_backend_invalidate_device_iotlb(dev, 0, -1ULL)) { + error_report("Fail to invalidate device iotlb"); + } + for (idx =3D 0; idx < dev->nvqs; ++idx) { + struct vhost_virtqueue *vq =3D dev->vqs + idx; + if (vhost_dev_has_iommu(dev) && + dev->vhost_ops->vhost_set_iotlb_callback) { + /* + * Update used ring information for IOTLB to work correctly, + * vhost-kernel code requires for this. + */ + vhost_device_iotlb_miss(dev, vq->used_phys, true); + } + vhost_shadow_vq_stop(dev, idx, dev->shadow_vqs[idx]); + vhost_virtqueue_start(dev, dev->vdev, &dev->vqs[idx], + dev->vq_index + idx); + } + + /* Enable guest's vq vring */ + r =3D dev->vhost_ops->vhost_dev_start(dev, true); + assert(r =3D=3D 0); + + for (idx =3D 0; idx < dev->nvqs; ++idx) { vhost_shadow_vq_free(dev->shadow_vqs[idx]); } =20 @@ -1236,9 +1274,64 @@ static int vhost_sw_live_migration_stop(struct vhost= _dev *dev) return 0; } =20 +/* + * Start shadow virtqueue in a given queue. + * In failure case, this function leaves queue working as regular vhost mo= de. + */ +static bool vhost_sw_live_migration_start_vq(struct vhost_dev *dev, + unsigned idx) +{ + struct vhost_vring_addr addr =3D { + .index =3D idx, + }; + struct vhost_vring_state s =3D { + .index =3D idx, + }; + int r; + bool ok; + + vhost_virtqueue_stop(dev, dev->vdev, &dev->vqs[idx], dev->vq_index + i= dx); + ok =3D vhost_shadow_vq_start(dev, idx, dev->shadow_vqs[idx]); + if (unlikely(!ok)) { + return false; + } + + /* From this point, vhost_virtqueue_start can reset these changes */ + vhost_shadow_vq_get_vring_addr(dev->shadow_vqs[idx], &addr); + r =3D dev->vhost_ops->vhost_set_vring_addr(dev, &addr); + if (unlikely(r !=3D 0)) { + VHOST_OPS_DEBUG("vhost_set_vring_addr for shadow vq failed"); + goto err; + } + + r =3D dev->vhost_ops->vhost_set_vring_base(dev, &s); + if (unlikely(r !=3D 0)) { + VHOST_OPS_DEBUG("vhost_set_vring_base for shadow vq failed"); + goto err; + } + + if (vhost_dev_has_iommu(dev) && dev->vhost_ops->vhost_set_iotlb_callba= ck) { + /* + * Update used ring information for IOTLB to work correctly, + * vhost-kernel code requires for this. + */ + r =3D vhost_device_iotlb_miss(dev, addr.used_user_addr, true); + if (unlikely(r !=3D 0)) { + /* Debug message already printed */ + goto err; + } + } + + return true; + +err: + vhost_virtqueue_start(dev, dev->vdev, &dev->vqs[idx], dev->vq_index + = idx); + return false; +} + static int vhost_sw_live_migration_start(struct vhost_dev *dev) { - int idx, stop_idx; + int r, idx, stop_idx; =20 dev->shadow_vqs =3D g_new0(VhostShadowVirtqueue *, dev->nvqs); for (idx =3D 0; idx < dev->nvqs; ++idx) { @@ -1248,23 +1341,37 @@ static int vhost_sw_live_migration_start(struct vho= st_dev *dev) } } =20 + r =3D dev->vhost_ops->vhost_vring_pause(dev); + assert(r =3D=3D 0); + if (vhost_backend_invalidate_device_iotlb(dev, 0, -1ULL)) { + error_report("Fail to invalidate device iotlb"); + } + + /* Can be read by vhost_virtqueue_mask, from vm exit */ dev->shadow_vqs_enabled =3D true; for (idx =3D 0; idx < dev->nvqs; ++idx) { - bool ok =3D vhost_shadow_vq_start(dev, idx, dev->shadow_vqs[idx]); + bool ok =3D vhost_sw_live_migration_start_vq(dev, idx); if (unlikely(!ok)) { goto err_start; } } =20 + /* Enable shadow vq vring */ + r =3D dev->vhost_ops->vhost_dev_start(dev, true); + assert(r =3D=3D 0); return 0; =20 err_start: dev->shadow_vqs_enabled =3D false; for (stop_idx =3D 0; stop_idx < idx; stop_idx++) { vhost_shadow_vq_stop(dev, idx, dev->shadow_vqs[stop_idx]); + vhost_virtqueue_start(dev, dev->vdev, &dev->vqs[idx], + dev->vq_index + stop_idx); } =20 err_new: + /* Enable guest's vring */ + dev->vhost_ops->vhost_set_vring_enable(dev, true); for (idx =3D 0; idx < dev->nvqs; ++idx) { vhost_shadow_vq_free(dev->shadow_vqs[idx]); } @@ -1979,6 +2086,27 @@ void qmp_x_vhost_enable_shadow_vq(const char *name, = bool enable, Error **errp) =20 if (!hdev->started) { err_cause =3D "Device is not started"; + } else if (!vhost_dev_has_iommu(hdev)) { + err_cause =3D "Does not support iommu"; + } else if (hdev->acked_features & BIT_ULL(VIRTIO_F_RING_PACKED)) { + err_cause =3D "Is packed"; + } else if (hdev->acked_features & BIT_ULL(VIRTIO_RING_F_EVENT_IDX)= ) { + err_cause =3D "Have event idx"; + } else if (hdev->acked_features & + BIT_ULL(VIRTIO_RING_F_INDIRECT_DESC)) { + err_cause =3D "Supports indirect descriptors"; + } else if (!hdev->vhost_ops->vhost_vring_pause || + !hdev->vhost_ops->vhost_dev_start) { + err_cause =3D "Cannot pause device"; + } else if (hdev->vhost_ops->vhost_get_iova_range) { + err_cause =3D "Device may not support all iova range"; + } else if (hdev->vhost_ops->vhost_enable_custom_iommu) { + err_cause =3D "Device does not use regular IOMMU"; + } else if (!virtio_vdev_has_feature(hdev->vdev, VIRTIO_F_VERSION_1= )) { + err_cause =3D "Legacy VirtIO device"; + } + + if (err_cause) { goto err; } =20 --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442591; cv=none; d=zohomail.com; s=zohoarc; b=YD9lrODYykPw43NPJtAotO5UGTte/nnwNTULXDQryn5AShMuj9P0zpoHmYVDKviFQTMnbxeKbPkQ7RWQqQkezjhOBwH/OES9JIDxe0j6fpDCANOuWJ3f2Mxz3QkRbWJ2lBX9OCm8JV+WoNZTlSKR2PGcZtgz8X7cdfNY6nmR7AM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442591; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=V9LrI5P9tyxXNbE1l58yZAifcj59RDu8GvtjXC0K7Og=; b=eEgg6Qk4/icoE9S+L9yjmiEhQmkTGuc+ie/seKw2Zmbvf5abI7+ZPw2iYnYU8/emoH1/GxrY7quW+UYJKE7r6YZtYrfipJYCD71m71pur0rtvwqcE4b7S4LkXWDxjvPB3rEHYqyM87bnqL5GDBwk13ADYeu/M9zWNrM4Ru12jgU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442591799785.6576585473198; Wed, 19 May 2021 09:43:11 -0700 (PDT) Received: from localhost ([::1]:55076 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPHr-0006Mk-02 for importer@patchew.org; Wed, 19 May 2021 12:43:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35098) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6h-0005vw-20 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:39 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:42824) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6e-0005vj-TV for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:38 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-273-83H4Bj-EPdO9aS9zczD4mQ-1; Wed, 19 May 2021 12:31:33 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9F607C73A4; Wed, 19 May 2021 16:31:31 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id A24495D6AC; Wed, 19 May 2021 16:31:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441896; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V9LrI5P9tyxXNbE1l58yZAifcj59RDu8GvtjXC0K7Og=; b=LcAqFqh2X7II5mS0+IPAK4TKFiTeAhxo7lCajMKLXbhmgYoE7gCR5w1fjAL1iM4knhxnVb Nq+sJ4bWG7efgL858jw3RkzgD9r89+yiG18w+pk7sRtsUxVzpnmh5rOE2EenFQ3aWBRi4/ HVbCUAN5Lg5y07vaKlVW1oyPQo/opWc= X-MC-Unique: 83H4Bj-EPdO9aS9zczD4mQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 18/29] vhost: Use vhost_enable_custom_iommu to unmap everything if available Date: Wed, 19 May 2021 18:28:52 +0200 Message-Id: <20210519162903.1172366-19-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This call is the right way to unmap every IOTLB in devices with non standard IOMMU (vdpa devices), since regular one would require an IOTLB message they don't support. Another possible solution would be to implement .vhost_send_device_iotlb_msg vhost operation in vhost-vdpa, but it could conflict with expected backend iommu operations. Currently, this method does not work for vp_vdpa. For some reason, intel IOMMU is not able to map anything when vdpa has unmapped everything. However that is on kernel side, this commit code should be as intended in the final version. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 5b5001a08a..c8fa9df9b3 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1241,7 +1241,12 @@ static int vhost_sw_live_migration_stop(struct vhost= _dev *dev) =20 r =3D dev->vhost_ops->vhost_vring_pause(dev); assert(r =3D=3D 0); - if (vhost_backend_invalidate_device_iotlb(dev, 0, -1ULL)) { + if (dev->vhost_ops->vhost_enable_custom_iommu) { + r =3D dev->vhost_ops->vhost_enable_custom_iommu(dev, false); + } else { + r =3D vhost_backend_invalidate_device_iotlb(dev, 0, -1ULL); + } + if (r) { error_report("Fail to invalidate device iotlb"); } =20 @@ -1343,7 +1348,12 @@ static int vhost_sw_live_migration_start(struct vhos= t_dev *dev) =20 r =3D dev->vhost_ops->vhost_vring_pause(dev); assert(r =3D=3D 0); - if (vhost_backend_invalidate_device_iotlb(dev, 0, -1ULL)) { + if (dev->vhost_ops->vhost_enable_custom_iommu) { + r =3D dev->vhost_ops->vhost_enable_custom_iommu(dev, false); + } else { + r =3D vhost_backend_invalidate_device_iotlb(dev, 0, -1ULL); + } + if (r) { error_report("Fail to invalidate device iotlb"); } =20 @@ -2100,8 +2110,6 @@ void qmp_x_vhost_enable_shadow_vq(const char *name, b= ool enable, Error **errp) err_cause =3D "Cannot pause device"; } else if (hdev->vhost_ops->vhost_get_iova_range) { err_cause =3D "Device may not support all iova range"; - } else if (hdev->vhost_ops->vhost_enable_custom_iommu) { - err_cause =3D "Device does not use regular IOMMU"; } else if (!virtio_vdev_has_feature(hdev->vdev, VIRTIO_F_VERSION_1= )) { err_cause =3D "Legacy VirtIO device"; } --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442876; cv=none; d=zohomail.com; s=zohoarc; b=HE5krfluMrj7CuSTQsEileOQzlN5QxGRQX/AogT7IFSdrSsiAS6PgaUQAGloVjY27qzp+Ux+SlnOQzPZ334+FUivnyNg+8fV4oP9HmjreYilBg/DGENLljIDswqVy7PemI+WAmyR5/VhzMWC0QwtKUG2g1qMGh7hDeEhJiUUqg4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442876; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ZO2vXSOcAYgTvZTq5H2SUet7zYlpax2GmAqSpPcJRYk=; b=H5QeZqb1OwL8zkRhN1Xh3mFgsHQXyJ0P6t3BnCXojZws43Lsxh3URbihJsDpoaPx/W1BcNqKkJVDKQsv7i5dovaPjhxyYBP1s6q5Zk2e4ONFBb1VEEZWvwTJFif07JrnP8WTpTnrfbnEwpHsdC42IRdJ9xkt6Lw/MjwxsIiAOwc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442876922351.68856384204594; Wed, 19 May 2021 09:47:56 -0700 (PDT) Received: from localhost ([::1]:35696 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPMR-000455-WB for importer@patchew.org; Wed, 19 May 2021 12:47:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35216) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP72-0006Fs-Bp for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:58414) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6p-00060U-KF for qemu-devel@nongnu.org; Wed, 19 May 2021 12:31:57 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-560-xwHAtvnaM5qWs_jpKoKhdg-1; Wed, 19 May 2021 12:31:45 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 49338180FD6D; Wed, 19 May 2021 16:31:44 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id F3EF45D6AC; Wed, 19 May 2021 16:31:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441907; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZO2vXSOcAYgTvZTq5H2SUet7zYlpax2GmAqSpPcJRYk=; b=BIz3LTB0+D9xJDfSJZzYpUJLcqc6eKzEyWon7H3YCX24u+tTyLTEdMFVY4soTqp9XoUoFN 8cRGSdQUdSG0mKUW0Cf7foM6Yl0wdy3NOjk+HRBWqpTVgFr90AlJrT5nElzT3VQ3ttaEdk wqjs0a59XYwYW2RumQpWjZOeMm0vw/k= X-MC-Unique: xwHAtvnaM5qWs_jpKoKhdg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 19/29] vhost: Check for device VRING_USED_F_NO_NOTIFY at shadow virtqueue kick Date: Wed, 19 May 2021 18:28:53 +0200 Message-Id: <20210519162903.1172366-20-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 6d767fe248..6b42147449 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -135,6 +135,15 @@ static void vhost_shadow_vq_add(VhostShadowVirtqueue *= svq, svq->ring_id_maps[qemu_head] =3D elem; } =20 +static void vhost_shadow_vq_kick(VhostShadowVirtqueue *svq) +{ + /* Make sure we are reading updated device flag */ + smp_rmb(); + if (!(svq->vring.used->flags & VRING_USED_F_NO_NOTIFY)) { + event_notifier_set(&svq->kick_notifier); + } +} + /* Handle guest->device notifications */ static void vhost_handle_guest_kick(EventNotifier *n) { @@ -159,7 +168,7 @@ static void vhost_handle_guest_kick(EventNotifier *n) } =20 vhost_shadow_vq_add(svq, elem); - event_notifier_set(&svq->kick_notifier); + vhost_shadow_vq_kick(svq); } =20 virtio_queue_set_notification(svq->vq, true); --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442921; cv=none; d=zohomail.com; s=zohoarc; b=DvrtEDAMRyeVB6MeZLrOEN5vjd6vhp0ETFrtmGdA11WpvQf3u53lj31w1v3VMZlNrrWrhKD7mRPZccrJgy+wPadAgmUV7owy01yLEKN0Qbnr5NmbM14/3sbHxEa0kgnx/nNU2lut728mFMGmir562VZWqP7bpkpDZAgYT+CgDMg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442921; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=pG7/gEG0Z91i9qQ6wBTx25rr6oZJeVJWvV5Jsi0WAfU=; b=F29AuAM2oBXwNNUvsCp44MJ6RVC4mpwOLwmH3X7mno64KYXYVEQlM6P4CW36qe4WwHkB2Er5bvNAi0yvmAu90IOWyQOybxIzrPWlTO+3ci6E/wl31k0gPIpPXyz07vQFH6TnIHtm0MvUdIuj9ltT0B20WFGuOf0YQTjdTDkwRX4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442921069786.5650303782189; Wed, 19 May 2021 09:48:41 -0700 (PDT) Received: from localhost ([::1]:38316 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPNA-0005qm-1M for importer@patchew.org; Wed, 19 May 2021 12:48:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35220) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP72-0006Ft-CJ for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:28837) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6y-00060h-CF for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:00 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-441-rFf2W_B9M-mGvDBX_Wi46g-1; Wed, 19 May 2021 12:31:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 93C2B10082E0; Wed, 19 May 2021 16:31:47 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9D5DA5D6AC; Wed, 19 May 2021 16:31:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441910; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pG7/gEG0Z91i9qQ6wBTx25rr6oZJeVJWvV5Jsi0WAfU=; b=AbEdTuS8VgDS63xAj9jf8dj5QTRYK+nYqfWgohiLeuvlyRNNQluZDE3lfUAipt4k6FxZ8c RT6b5regT7v+lqhtOdTChimfM7OBpAP2oPYkh8knfeJjBAJEMMwmE/cAVwl+rckNBYr8Nl QU0mgQtyXQP2+EophFS/hgpAoM0eUOc= X-MC-Unique: rFf2W_B9M-mGvDBX_Wi46g-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 20/29] vhost: Use VRING_AVAIL_F_NO_INTERRUPT at device call on shadow virtqueue Date: Wed, 19 May 2021 18:28:54 +0200 Message-Id: <20210519162903.1172366-21-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 6b42147449..934d3bb27b 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -64,8 +64,30 @@ typedef struct VhostShadowVirtqueue { =20 /* Next head to consume from device */ uint16_t used_idx; + + /* Cache for the exposed notification flag */ + bool notification; } VhostShadowVirtqueue; =20 +static void vhost_shadow_vq_set_notification(VhostShadowVirtqueue *svq, + bool enable) +{ + uint16_t notification_flag; + + if (svq->notification =3D=3D enable) { + return; + } + + notification_flag =3D cpu_to_le16(VRING_AVAIL_F_NO_INTERRUPT); + + svq->notification =3D enable; + if (enable) { + svq->vring.avail->flags &=3D ~notification_flag; + } else { + svq->vring.avail->flags |=3D notification_flag; + } +} + static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, const struct iovec *iovec, size_t num, bool more_descs, bool writ= e) @@ -231,7 +253,7 @@ static void vhost_shadow_vq_handle_call_no_test(EventNo= tifier *n) do { unsigned i =3D 0; =20 - /* TODO: Use VRING_AVAIL_F_NO_INTERRUPT */ + vhost_shadow_vq_set_notification(svq, false); while (true) { g_autofree VirtQueueElement *elem =3D vhost_shadow_vq_get_buf(= svq); if (!elem) { @@ -249,6 +271,7 @@ static void vhost_shadow_vq_handle_call_no_test(EventNo= tifier *n) svq->masked_notifier.signaled =3D true; event_notifier_set(svq->masked_notifier.n); } + vhost_shadow_vq_set_notification(svq, true); } while (vhost_shadow_vq_more_used(svq)); } =20 --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621443061; cv=none; d=zohomail.com; s=zohoarc; b=IYjNpLyBuqYeO2v55SF/AS7ykv75kuugXDclhv5Iqa21D+0zDO2P/DB5bQ35ovT3oEYHGIb21E1QEnephMj/n01+IwcG7ng8Lr3VenOcftNkSqo++EKS+Bwe9c9TM9TnoTalyYFQnf9TcthUJiVgDkDX10A8tn9EGoQvqgjRsMs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621443061; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ISvBgRyvzwXN9OxWJU0DoZshENCWFp3ZYOXUMnTaqU0=; b=DKMbEBWNb1WUKeO8USh8yRcpE9sovEuzx2dExtpDZfAjGXXN+FpiA2c2IVf1510wWTy6wQRVHs8PneeDXddW1XdBEJQDiu8TxIC4yeavG4wtcVyLGDzzH8wtiZ/sDpsP7MqqwVukxFrtFtkic/BZg9bgNUIJeq+ZR/f7So6G7mM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621443061937664.3373930344287; Wed, 19 May 2021 09:51:01 -0700 (PDT) Received: from localhost ([::1]:44432 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPPR-0001l5-5x for importer@patchew.org; Wed, 19 May 2021 12:51:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35288) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP78-0006Ia-2d for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:57529) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP6z-00061h-70 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:05 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-255-1EX-cyr8OVGeZZPXXSVE5A-1; Wed, 19 May 2021 12:31:52 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DC4D7180FD6B; Wed, 19 May 2021 16:31:50 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id E94555D6AC; Wed, 19 May 2021 16:31:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441914; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ISvBgRyvzwXN9OxWJU0DoZshENCWFp3ZYOXUMnTaqU0=; b=Z55/d0SZKsq0IfkTm/w31LO9zeL21En+phCqp48FHEtBN5SXuy8dRGsfBQ7DbPLi+SbKfQ AoDvQ/T+m0Rf3Wm3hrZsVQ+03/W9UaEo9gTruT+Z4WXUoJxwgPU4IMBOdmanpwWGPDIsyU 4Oe9vUqXXb36JUgWKPVByGdSDmQLOSk= X-MC-Unique: 1EX-cyr8OVGeZZPXXSVE5A-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 21/29] vhost: Add VhostIOVATree Date: Wed, 19 May 2021 18:28:55 +0200 Message-Id: <20210519162903.1172366-22-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This tree is able to look for a translated address from a IOVA address. At first glance is similar to util/iova-tree. However, SVQ working on devices with limited IOVA space need more capabilities, like allocating IOVA chunks or perform reverse translations (qemu addresses to iova). Starting a sepparated implementation. Knowing than insertions/deletions will not be as frequent as searches, it uses an ordered array at implementation. A different name could be used, but ordered searchable array is a little bit long though. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-iova-tree.h | 50 ++++++++++ hw/virtio/vhost-iova-tree.c | 188 ++++++++++++++++++++++++++++++++++++ hw/virtio/meson.build | 2 +- 3 files changed, 239 insertions(+), 1 deletion(-) create mode 100644 hw/virtio/vhost-iova-tree.h create mode 100644 hw/virtio/vhost-iova-tree.c diff --git a/hw/virtio/vhost-iova-tree.h b/hw/virtio/vhost-iova-tree.h new file mode 100644 index 0000000000..2a44af8b3a --- /dev/null +++ b/hw/virtio/vhost-iova-tree.h @@ -0,0 +1,50 @@ +/* + * vhost software live migration ring + * + * SPDX-FileCopyrightText: Red Hat, Inc. 2021 + * SPDX-FileContributor: Author: Eugenio P=C3=A9rez + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef HW_VIRTIO_VHOST_IOVA_TREE_H +#define HW_VIRTIO_VHOST_IOVA_TREE_H + +#include + +#include "exec/memory.h" + +typedef struct VhostDMAMap { + void *translated_addr; + hwaddr iova; + hwaddr size; /* Inclusive */ + IOMMUAccessFlags perm; +} VhostDMAMap; + +typedef enum VhostDMAMapNewRC { + VHOST_DMA_MAP_OVERLAP =3D -2, + VHOST_DMA_MAP_INVALID =3D -1, + VHOST_DMA_MAP_OK =3D 0, +} VhostDMAMapNewRC; + +/** + * VhostIOVATree + * + * Store and search IOVA -> Translated mappings. + * + * Note that it cannot remove nodes. + */ +typedef struct VhostIOVATree { + /* Ordered array of reverse translations, IOVA address to qemu memory.= */ + GArray *iova_taddr_map; +} VhostIOVATree; + +void vhost_iova_tree_new(VhostIOVATree *iova_rm); +void vhost_iova_tree_destroy(VhostIOVATree *iova_rm); + +const VhostDMAMap *vhost_iova_tree_find_taddr(const VhostIOVATree *iova_rm, + const VhostDMAMap *map); +VhostDMAMapNewRC vhost_iova_tree_insert(VhostIOVATree *iova_rm, + VhostDMAMap *map); + +#endif diff --git a/hw/virtio/vhost-iova-tree.c b/hw/virtio/vhost-iova-tree.c new file mode 100644 index 0000000000..dfd7e448b5 --- /dev/null +++ b/hw/virtio/vhost-iova-tree.c @@ -0,0 +1,188 @@ +/* + * vhost software live migration ring + * + * SPDX-FileCopyrightText: Red Hat, Inc. 2021 + * SPDX-FileContributor: Author: Eugenio P=C3=A9rez + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "vhost-iova-tree.h" + +#define G_ARRAY_NOT_ZERO_TERMINATED false +#define G_ARRAY_NOT_CLEAR_ON_ALLOC false + +/** + * Inserts an element after an existing one in garray. + * + * @array The array + * @prev_elem The previous element of array of NULL if prepending + * @map The DMA map + * + * It provides the aditional advantage of being type safe over + * g_array_insert_val, which accepts a reference pointer instead of a value + * with no complains. + */ +static void vhost_iova_tree_insert_after(GArray *array, + const VhostDMAMap *prev_elem, + const VhostDMAMap *map) +{ + size_t pos; + + if (!prev_elem) { + pos =3D 0; + } else { + pos =3D prev_elem - &g_array_index(array, typeof(*prev_elem), 0) += 1; + } + + g_array_insert_val(array, pos, *map); +} + +static gint vhost_iova_tree_cmp_iova(gconstpointer a, gconstpointer b) +{ + const VhostDMAMap *m1 =3D a, *m2 =3D b; + + if (m1->iova > m2->iova + m2->size) { + return 1; + } + + if (m1->iova + m1->size < m2->iova) { + return -1; + } + + /* Overlapped */ + return 0; +} + +/** + * Find the previous node to a given iova + * + * @array The ascending ordered-by-translated-addr array of VhostDMAMap + * @map The map to insert + * @prev Returned location of the previous map + * + * Return VHOST_DMA_MAP_OK if everything went well, or VHOST_DMA_MAP_OVERL= AP if + * it already exists. It is ok to use this function to check if a given ra= nge + * exists, but it will use a linear search. + * + * TODO: We can use bsearch to locate the entry if we save the state in the + * needle, knowing that the needle is always the first argument to + * compare_func. + */ +static VhostDMAMapNewRC vhost_iova_tree_find_prev(const GArray *array, + GCompareFunc compare_fun= c, + const VhostDMAMap *map, + const VhostDMAMap **prev) +{ + size_t i; + int r; + + *prev =3D NULL; + for (i =3D 0; i < array->len; ++i) { + r =3D compare_func(map, &g_array_index(array, typeof(*map), i)); + if (r =3D=3D 0) { + return VHOST_DMA_MAP_OVERLAP; + } + if (r < 0) { + return VHOST_DMA_MAP_OK; + } + + *prev =3D &g_array_index(array, typeof(**prev), i); + } + + return VHOST_DMA_MAP_OK; +} + +/** + * Create a new IOVA tree + * + * @tree The IOVA tree + */ +void vhost_iova_tree_new(VhostIOVATree *tree) +{ + assert(tree); + + tree->iova_taddr_map =3D g_array_new(G_ARRAY_NOT_ZERO_TERMINATED, + G_ARRAY_NOT_CLEAR_ON_ALLOC, + sizeof(VhostDMAMap)); +} + +/** + * Destroy an IOVA tree + * + * @tree The iova tree + */ +void vhost_iova_tree_destroy(VhostIOVATree *tree) +{ + g_array_unref(g_steal_pointer(&tree->iova_taddr_map)); +} + +/** + * Perform a search on a GArray. + * + * @array Glib array + * @map Map to look up + * @compare_func Compare function to use + * + * Return The found element or NULL if not found. + * + * This can be replaced with g_array_binary_search (Since glib 2.62) when = that + * is common enough. + */ +static const VhostDMAMap *vhost_iova_tree_bsearch(const GArray *array, + const VhostDMAMap *map, + GCompareFunc compare_fun= c) +{ + return bsearch(map, array->data, array->len, sizeof(*map), compare_fun= c); +} + +/** + * Find the translated address stored from a IOVA address + * + * @tree The iova tree + * @map The map with the memory address + * + * Return the stored mapping, or NULL if not found. + */ +const VhostDMAMap *vhost_iova_tree_find_taddr(const VhostIOVATree *tree, + const VhostDMAMap *map) +{ + return vhost_iova_tree_bsearch(tree->iova_taddr_map, map, + vhost_iova_tree_cmp_iova); +} + +/** + * Insert a new map + * + * @tree The iova tree + * @map The iova map + * + * Returns: + * - VHOST_DMA_MAP_OK if the map fits in the container + * - VHOST_DMA_MAP_INVALID if the map does not make sense (like size overf= low) + * - VHOST_DMA_MAP_OVERLAP if the tree already contains that map + * Can query the assignated iova in map. + */ +VhostDMAMapNewRC vhost_iova_tree_insert(VhostIOVATree *tree, + VhostDMAMap *map) +{ + const VhostDMAMap *prev; + int find_prev_rc; + + if (map->translated_addr + map->size < map->translated_addr || + map->iova + map->size < map->iova || map->perm =3D=3D IOMMU_NONE) { + return VHOST_DMA_MAP_INVALID; + } + + /* Check for duplicates, and save position for insertion */ + find_prev_rc =3D vhost_iova_tree_find_prev(tree->iova_taddr_map, + vhost_iova_tree_cmp_iova, map, + &prev); + if (find_prev_rc =3D=3D VHOST_DMA_MAP_OVERLAP) { + return VHOST_DMA_MAP_OVERLAP; + } + + vhost_iova_tree_insert_after(tree->iova_taddr_map, prev, map); + return VHOST_DMA_MAP_OK; +} diff --git a/hw/virtio/meson.build b/hw/virtio/meson.build index 8b5a0225fe..cb306b83c6 100644 --- a/hw/virtio/meson.build +++ b/hw/virtio/meson.build @@ -11,7 +11,7 @@ softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost-= stub.c')) =20 virtio_ss =3D ss.source_set() virtio_ss.add(files('virtio.c')) -virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backe= nd.c', 'vhost-shadow-virtqueue.c')) +virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backe= nd.c', 'vhost-shadow-virtqueue.c', 'vhost-iova-tree.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER', if_true: files('vhost-user.c')) virtio_ss.add(when: 'CONFIG_VHOST_VDPA', if_true: files('vhost-vdpa.c')) virtio_ss.add(when: 'CONFIG_VIRTIO_BALLOON', if_true: files('virtio-balloo= n.c')) --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621443217; cv=none; d=zohomail.com; s=zohoarc; b=jRqOPO6EfgXRqQz8ZTyVkQDxTEOTQLCDGoCBpv81COyMCtzZiY+Qq4BY3a3fq4qvJZwy7rMc+zb/B0fyjquK+4OZVqehedJjAREfc5XnvvWB451xMkseKrZPBpUNpbf9muOTHydbeg7UTRRdxPlAK4mKEWQVEVEtKhqHnN7wvTw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621443217; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=uP4UtvtAVqjJFYeWJ8J8gGnTPjK6WSRqP+XvyKxXAtY=; b=U/ROKo85eUg3V3XzXzw8WBSDTL4yPaz9qx4mw4RWIdrpO2yFobJHJBlWEElxbrqUxCkp3SVMmBiTEryiW6+7rNz9uusy1vRNb2gp5JwJbJrMFzQ3W0CyDemkt70YrHolW7TJrTiSWgCpTi8hd0Y4yXX0nQvxlnvmToyQuu8YB8s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621443217397412.00789993695594; Wed, 19 May 2021 09:53:37 -0700 (PDT) Received: from localhost ([::1]:53142 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPRw-0007hY-HU for importer@patchew.org; Wed, 19 May 2021 12:53:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35308) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7B-0006Ov-Fe for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:53023) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP72-00062D-7U for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:09 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-181-pXOuTORTNKqDLRTZ1BdAgA-1; Wed, 19 May 2021 12:31:56 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6E030107ACC7; Wed, 19 May 2021 16:31:54 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3C43F5D6AC; Wed, 19 May 2021 16:31:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441919; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uP4UtvtAVqjJFYeWJ8J8gGnTPjK6WSRqP+XvyKxXAtY=; b=dio4liD2o2WwwhJKk7Ub+RtXB2TwgES1trRkYHhR5beaKQg7PINuyX8wKhrvymwBORu00p Kpzqh4odqhqSaw0NsaHJNCFzk6tGfQ419xNVQGA46KVzf6YwRkUCzWvGDDRBG6UM5dLGlY q3ZicqdEmJ/YnbluEGYDHHoXf7/fGEA= X-MC-Unique: pXOuTORTNKqDLRTZ1BdAgA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 22/29] vhost: Add iova_rev_maps_find_iova to IOVAReverseMaps Date: Wed, 19 May 2021 18:28:56 +0200 Message-Id: <20210519162903.1172366-23-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Shadow virtqueue can translate addresses from guest's address to it's own address space this way. It duplicates the array so it can search efficiently both directions, and it will signal overlap if iova or the translated address is present in it's each array. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-iova-tree.h | 10 +++++++- hw/virtio/vhost-iova-tree.c | 49 ++++++++++++++++++++++++++++++++++--- 2 files changed, 55 insertions(+), 4 deletions(-) diff --git a/hw/virtio/vhost-iova-tree.h b/hw/virtio/vhost-iova-tree.h index 2a44af8b3a..589e86bd91 100644 --- a/hw/virtio/vhost-iova-tree.h +++ b/hw/virtio/vhost-iova-tree.h @@ -30,18 +30,26 @@ typedef enum VhostDMAMapNewRC { /** * VhostIOVATree * - * Store and search IOVA -> Translated mappings. + * Store and search IOVA -> Translated mappings and the reverse, from + * translated address to IOVA. * * Note that it cannot remove nodes. */ typedef struct VhostIOVATree { /* Ordered array of reverse translations, IOVA address to qemu memory.= */ GArray *iova_taddr_map; + + /* + * Ordered array of translations from qemu virtual memory address to i= ova + */ + GArray *taddr_iova_map; } VhostIOVATree; =20 void vhost_iova_tree_new(VhostIOVATree *iova_rm); void vhost_iova_tree_destroy(VhostIOVATree *iova_rm); =20 +const VhostDMAMap *vhost_iova_tree_find_iova(const VhostIOVATree *iova_rm, + const VhostDMAMap *map); const VhostDMAMap *vhost_iova_tree_find_taddr(const VhostIOVATree *iova_rm, const VhostDMAMap *map); VhostDMAMapNewRC vhost_iova_tree_insert(VhostIOVATree *iova_rm, diff --git a/hw/virtio/vhost-iova-tree.c b/hw/virtio/vhost-iova-tree.c index dfd7e448b5..2900390a1e 100644 --- a/hw/virtio/vhost-iova-tree.c +++ b/hw/virtio/vhost-iova-tree.c @@ -39,6 +39,22 @@ static void vhost_iova_tree_insert_after(GArray *array, g_array_insert_val(array, pos, *map); } =20 +static gint vhost_iova_tree_cmp_taddr(gconstpointer a, gconstpointer b) +{ + const VhostDMAMap *m1 =3D a, *m2 =3D b; + + if (m1->translated_addr > m2->translated_addr + m2->size) { + return 1; + } + + if (m1->translated_addr + m1->size < m2->translated_addr) { + return -1; + } + + /* Overlapped */ + return 0; +} + static gint vhost_iova_tree_cmp_iova(gconstpointer a, gconstpointer b) { const VhostDMAMap *m1 =3D a, *m2 =3D b; @@ -106,6 +122,9 @@ void vhost_iova_tree_new(VhostIOVATree *tree) tree->iova_taddr_map =3D g_array_new(G_ARRAY_NOT_ZERO_TERMINATED, G_ARRAY_NOT_CLEAR_ON_ALLOC, sizeof(VhostDMAMap)); + tree->taddr_iova_map =3D g_array_new(G_ARRAY_NOT_ZERO_TERMINATED, + G_ARRAY_NOT_CLEAR_ON_ALLOC, + sizeof(VhostDMAMap)); } =20 /** @@ -116,6 +135,7 @@ void vhost_iova_tree_new(VhostIOVATree *tree) void vhost_iova_tree_destroy(VhostIOVATree *tree) { g_array_unref(g_steal_pointer(&tree->iova_taddr_map)); + g_array_unref(g_steal_pointer(&tree->taddr_iova_map)); } =20 /** @@ -137,6 +157,21 @@ static const VhostDMAMap *vhost_iova_tree_bsearch(cons= t GArray *array, return bsearch(map, array->data, array->len, sizeof(*map), compare_fun= c); } =20 +/** + * Find the IOVA address stored from a memory address + * + * @tree The iova tree + * @map The map with the memory address + * + * Return the stored mapping, or NULL if not found. + */ +const VhostDMAMap *vhost_iova_tree_find_iova(const VhostIOVATree *tree, + const VhostDMAMap *map) +{ + return vhost_iova_tree_bsearch(tree->taddr_iova_map, map, + vhost_iova_tree_cmp_taddr); +} + /** * Find the translated address stored from a IOVA address * @@ -167,7 +202,7 @@ const VhostDMAMap *vhost_iova_tree_find_taddr(const Vho= stIOVATree *tree, VhostDMAMapNewRC vhost_iova_tree_insert(VhostIOVATree *tree, VhostDMAMap *map) { - const VhostDMAMap *prev; + const VhostDMAMap *qemu_prev, *iova_prev; int find_prev_rc; =20 if (map->translated_addr + map->size < map->translated_addr || @@ -178,11 +213,19 @@ VhostDMAMapNewRC vhost_iova_tree_insert(VhostIOVATree= *tree, /* Check for duplicates, and save position for insertion */ find_prev_rc =3D vhost_iova_tree_find_prev(tree->iova_taddr_map, vhost_iova_tree_cmp_iova, map, - &prev); + &iova_prev); + if (find_prev_rc =3D=3D VHOST_DMA_MAP_OVERLAP) { + return VHOST_DMA_MAP_OVERLAP; + } + + find_prev_rc =3D vhost_iova_tree_find_prev(tree->taddr_iova_map, + vhost_iova_tree_cmp_taddr, ma= p, + &qemu_prev); if (find_prev_rc =3D=3D VHOST_DMA_MAP_OVERLAP) { return VHOST_DMA_MAP_OVERLAP; } =20 - vhost_iova_tree_insert_after(tree->iova_taddr_map, prev, map); + vhost_iova_tree_insert_after(tree->iova_taddr_map, iova_prev, map); + vhost_iova_tree_insert_after(tree->taddr_iova_map, qemu_prev, map); return VHOST_DMA_MAP_OK; } --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621443400; cv=none; d=zohomail.com; s=zohoarc; b=boAJA9qi/eaTYePsbsq8uD1lqim/OvN3ZsZLznERtVNn38+sGVu57wDL7YyKqjMyh3RjO0djWExNk0SujGNPwfvMT56y8kUvRowjfccZM5ko0neP7CJQ1qyYg2FMR768P9PxGhTXmYbZ4bWaPSmAIdgDnhrassDE3ZROcvqVV5Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621443400; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=3EN2RtzwvYrUtWholmNU/bVJLAfkhNFdmmfMWBtVVPE=; b=XItfWoNIOsH01RtJZISPrO6klxtMKdz3X0yjM2pecjAeK4eCS4j3yn2ES6XLp+8Bbyvkv/WE1VhuWYLs+xvatiY4VKus9r9TKdzT549DERN54yoo9fnKvPmV0rAStWhxuvIlw5//m9fAIYtZXNNheUXr4kBY8UExCNE/bdiaOtQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621443400352703.7531450673218; Wed, 19 May 2021 09:56:40 -0700 (PDT) Received: from localhost ([::1]:35004 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPUt-00062P-10 for importer@patchew.org; Wed, 19 May 2021 12:56:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35342) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7F-0006dp-Iy for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:48142) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7C-00064r-JD for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:13 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-407-9KOtJN-6Nf6rUjMRoktSIA-1; Wed, 19 May 2021 12:32:08 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BE60F107ACC7; Wed, 19 May 2021 16:32:06 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id C205B5D6AC; Wed, 19 May 2021 16:31:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441930; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3EN2RtzwvYrUtWholmNU/bVJLAfkhNFdmmfMWBtVVPE=; b=UiRi7O0MaoTO7YnbL951I3Ph1ORYNhdU38JsmO50vBNI8rbHz3id2aXUK5nUxtjdxm9XK4 qp494CU1JlzU8M2/EN5b78xlWPGEOxrPuOLnObqeGTYlP8SBCiV47Cv2nQ2pJz9y5wbCPI Wvy2Qh0DJABy/o800PZzpoDwVsXOh+0= X-MC-Unique: 9KOtJN-6Nf6rUjMRoktSIA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 23/29] vhost: Use a tree to store memory mappings Date: Wed, 19 May 2021 18:28:57 +0200 Message-Id: <20210519162903.1172366-24-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) At the moment, the tree is only used to store 1:1 maps of the qemu virtual addresses of shadow virtqueue vring and the guest's addresses. In other words, the tree only serves to check if the address the guest exposed is valid at the moment qemu receives the miss. It does not work if device has restrictions in its iova range at the moment. Updates to tree are protected by BQL, each one always run from main event loop context. vhost_device_iotlb_miss runs in the same one on reading it. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost.h | 3 + hw/virtio/vhost.c | 121 ++++++++++++++++++++++++++++++-------- 2 files changed, 99 insertions(+), 25 deletions(-) diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index c97a4c0017..773f882145 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -2,6 +2,7 @@ #define VHOST_H =20 #include "hw/virtio/vhost-backend.h" +#include "hw/virtio/vhost-iova-tree.h" #include "hw/virtio/virtio.h" #include "exec/memory.h" =20 @@ -88,6 +89,8 @@ struct vhost_dev { bool log_enabled; bool shadow_vqs_enabled; uint64_t log_size; + /* IOVA mapping used by Shadow Virtqueue */ + VhostIOVATree iova_map; struct { hwaddr first; hwaddr last; diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index c8fa9df9b3..925d2146a4 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1013,31 +1013,45 @@ static int vhost_memory_region_lookup(struct vhost_= dev *hdev, =20 int vhost_device_iotlb_miss(struct vhost_dev *dev, uint64_t iova, int writ= e) { - IOMMUTLBEntry iotlb; + IOMMUAccessFlags perm; uint64_t uaddr, len; int ret =3D -EFAULT; =20 - RCU_READ_LOCK_GUARD(); - trace_vhost_iotlb_miss(dev, 1); =20 if (dev->shadow_vqs_enabled) { - uaddr =3D iova; - len =3D 4096; - ret =3D vhost_backend_update_device_iotlb(dev, iova, uaddr, len, - IOMMU_RW); - if (ret) { - trace_vhost_iotlb_miss(dev, 2); - error_report("Fail to update device iotlb"); + /* Shadow virtqueue translations in its Virtual Address Space */ + const VhostDMAMap *result; + const VhostDMAMap needle =3D { + .iova =3D iova, + }; + + result =3D vhost_iova_tree_find_taddr(&dev->iova_map, &needle); + + if (unlikely(!result)) { + goto out; } =20 - return ret; - } + iova =3D result->iova; + uaddr =3D (uint64_t)result->translated_addr; + /* + * In IOVATree, result.iova + result.size is the last element of i= ova. + * For vhost, it is one past that last element. + */ + len =3D result->size + 1; + perm =3D result->perm; + } else { + IOMMUTLBEntry iotlb; + + RCU_READ_LOCK_GUARD(); + iotlb =3D address_space_get_iotlb_entry(dev->vdev->dma_as, + iova, write, + MEMTXATTRS_UNSPECIFIED); + + if (iotlb.target_as =3D=3D NULL) { + goto out; + } =20 - iotlb =3D address_space_get_iotlb_entry(dev->vdev->dma_as, - iova, write, - MEMTXATTRS_UNSPECIFIED); - if (iotlb.target_as !=3D NULL) { ret =3D vhost_memory_region_lookup(dev, iotlb.translated_addr, &uaddr, &len); if (ret) { @@ -1049,14 +1063,14 @@ int vhost_device_iotlb_miss(struct vhost_dev *dev, = uint64_t iova, int write) =20 len =3D MIN(iotlb.addr_mask + 1, len); iova =3D iova & ~iotlb.addr_mask; + perm =3D iotlb.perm; + } =20 - ret =3D vhost_backend_update_device_iotlb(dev, iova, uaddr, - len, iotlb.perm); - if (ret) { - trace_vhost_iotlb_miss(dev, 4); - error_report("Fail to update device iotlb"); - goto out; - } + ret =3D vhost_backend_update_device_iotlb(dev, iova, uaddr, len, perm); + if (ret) { + trace_vhost_iotlb_miss(dev, 4); + error_report("Fail to update device iotlb"); + goto out; } =20 trace_vhost_iotlb_miss(dev, 2); @@ -1249,7 +1263,7 @@ static int vhost_sw_live_migration_stop(struct vhost_= dev *dev) if (r) { error_report("Fail to invalidate device iotlb"); } - + vhost_iova_tree_destroy(&dev->iova_map); for (idx =3D 0; idx < dev->nvqs; ++idx) { struct vhost_virtqueue *vq =3D dev->vqs + idx; if (vhost_dev_has_iommu(dev) && @@ -1279,6 +1293,26 @@ static int vhost_sw_live_migration_stop(struct vhost= _dev *dev) return 0; } =20 +static bool vhost_shadow_vq_start_store_sections(struct vhost_dev *dev) +{ + int idx; + + for (idx =3D 0; idx < dev->n_mem_sections; ++idx) { + size_t region_size =3D dev->mem->regions[idx].memory_size; + VhostDMAMap region =3D { + .iova =3D dev->mem->regions[idx].userspace_addr, + .translated_addr =3D (void *)dev->mem->regions[idx].userspace_= addr, + .size =3D region_size - 1, + .perm =3D VHOST_ACCESS_RW, + }; + + VhostDMAMapNewRC r =3D vhost_iova_tree_insert(&dev->iova_map, ®= ion); + assert(r =3D=3D VHOST_DMA_MAP_OK); + } + + return true; +} + /* * Start shadow virtqueue in a given queue. * In failure case, this function leaves queue working as regular vhost mo= de. @@ -1292,9 +1326,37 @@ static bool vhost_sw_live_migration_start_vq(struct = vhost_dev *dev, struct vhost_vring_state s =3D { .index =3D idx, }; + VhostDMAMap driver_region, device_region; + int r; bool ok; =20 + assert(dev->shadow_vqs[idx] !=3D NULL); + vhost_shadow_vq_get_vring_addr(dev->shadow_vqs[idx], &addr); + driver_region =3D (VhostDMAMap) { + .iova =3D addr.desc_user_addr, + .translated_addr =3D (void *)addr.desc_user_addr, + + /* + * DMAMAp.size include the last byte included in the range, while + * sizeof marks one past it. Substract one byte to make them match. + */ + .size =3D vhost_shadow_vq_driver_area_size(dev->shadow_vqs[idx]) -= 1, + .perm =3D VHOST_ACCESS_RO, + }; + device_region =3D (VhostDMAMap) { + .iova =3D addr.used_user_addr, + .translated_addr =3D (void *)addr.used_user_addr, + .size =3D vhost_shadow_vq_device_area_size(dev->shadow_vqs[idx]) -= 1, + .perm =3D VHOST_ACCESS_RW, + }; + + r =3D vhost_iova_tree_insert(&dev->iova_map, &driver_region); + assert(r =3D=3D VHOST_DMA_MAP_OK); + + r =3D vhost_iova_tree_insert(&dev->iova_map, &device_region); + assert(r =3D=3D VHOST_DMA_MAP_OK); + vhost_virtqueue_stop(dev, dev->vdev, &dev->vqs[idx], dev->vq_index + i= dx); ok =3D vhost_shadow_vq_start(dev, idx, dev->shadow_vqs[idx]); if (unlikely(!ok)) { @@ -1302,7 +1364,6 @@ static bool vhost_sw_live_migration_start_vq(struct v= host_dev *dev, } =20 /* From this point, vhost_virtqueue_start can reset these changes */ - vhost_shadow_vq_get_vring_addr(dev->shadow_vqs[idx], &addr); r =3D dev->vhost_ops->vhost_set_vring_addr(dev, &addr); if (unlikely(r !=3D 0)) { VHOST_OPS_DEBUG("vhost_set_vring_addr for shadow vq failed"); @@ -1315,6 +1376,7 @@ static bool vhost_sw_live_migration_start_vq(struct v= host_dev *dev, goto err; } =20 + if (vhost_dev_has_iommu(dev) && dev->vhost_ops->vhost_set_iotlb_callba= ck) { /* * Update used ring information for IOTLB to work correctly, @@ -1357,6 +1419,15 @@ static int vhost_sw_live_migration_start(struct vhos= t_dev *dev) error_report("Fail to invalidate device iotlb"); } =20 + /* + * Create new iova mappings. SVQ always expose qemu's VA. + * TODO: Fine tune the exported mapping. Default vhost does not expose + * everything. + */ + + vhost_iova_tree_new(&dev->iova_map); + vhost_shadow_vq_start_store_sections(dev); + /* Can be read by vhost_virtqueue_mask, from vm exit */ dev->shadow_vqs_enabled =3D true; for (idx =3D 0; idx < dev->nvqs; ++idx) { --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621443154; cv=none; d=zohomail.com; s=zohoarc; b=m6ukT8ojlGC1sk7Ut4ZLKevr0Kc+TsR3XO7GV7/6qN7BJ1gS5Y+6uu+IaGCDHUJMtfp0fOHE06A6MhDTrbbF3sdutj0R06TlBzeNiGa4KVlO4Xkvr+9GeLGcs9lqSkIbwAlebV17OOPkiTXSUt7KbhxbACPYtXDbPu7mRkR6xOU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621443154; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=r7dtTOJdCmRYBm6EIMcBw+fVvfbwCuQ3tsienHtyc0Y=; b=Sw286+Er+MReFKS43lEK5RPzBwsoMLnoa2thD0tV/pUD3u6qBBRQLaHui2IcesUnfoyKnTCrAUQ27i1hgEYG/308aXvZDIG4mDMiQqUAt09u4qHFC2IbHTvrNg80aDg+pWK/GaUcR63Q2WHP90DqQyLlhxmpnS3s7cYABD+afZY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621443154767955.0603189279402; Wed, 19 May 2021 09:52:34 -0700 (PDT) Received: from localhost ([::1]:48942 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPQv-0004mI-FN for importer@patchew.org; Wed, 19 May 2021 12:52:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35372) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7J-0006qp-50 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:60980) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7H-000672-8U for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:16 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-34-V6j5Gc9oPmWp9KUswrXCXg-1; Wed, 19 May 2021 12:32:11 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 136ECC73A0; Wed, 19 May 2021 16:32:10 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1EFB55D6AC; Wed, 19 May 2021 16:32:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441934; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r7dtTOJdCmRYBm6EIMcBw+fVvfbwCuQ3tsienHtyc0Y=; b=bv3gNVA5GqJwbonOU3h9BctyrWVr5BZMU6KCDxPcNZqBiDaVLkPBnhJHfmwCWEbr0micFS zW+vnJQefVSB9UwGZwKcardw/oZPZY1qQZnb8mDT+vX2OaJJBsUTWNA/SOOL6CoZbcdvVG FCgOvdFjpQi/PwXVn0B3tIiyrwxyIEs= X-MC-Unique: V6j5Gc9oPmWp9KUswrXCXg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 24/29] vhost: Add iova_rev_maps_alloc Date: Wed, 19 May 2021 18:28:58 +0200 Message-Id: <20210519162903.1172366-25-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This way the tree can store/map a completely new virtual address space, in the range the device admits. It does not limit the range it will allocate, but the IOVA address of the maps are allocated growing. A range limitation will be need for the cases where start_addr !=3D 0. Tools for remove mappings will be needed also. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-iova-tree.h | 11 +++--- hw/virtio/vhost-iova-tree.c | 72 +++++++++++++++++++++++++++++++------ 2 files changed, 69 insertions(+), 14 deletions(-) diff --git a/hw/virtio/vhost-iova-tree.h b/hw/virtio/vhost-iova-tree.h index 589e86bd91..92e2d992b3 100644 --- a/hw/virtio/vhost-iova-tree.h +++ b/hw/virtio/vhost-iova-tree.h @@ -22,16 +22,17 @@ typedef struct VhostDMAMap { } VhostDMAMap; =20 typedef enum VhostDMAMapNewRC { + VHOST_DMA_MAP_NO_SPACE =3D -3, VHOST_DMA_MAP_OVERLAP =3D -2, VHOST_DMA_MAP_INVALID =3D -1, VHOST_DMA_MAP_OK =3D 0, } VhostDMAMapNewRC; =20 /** - * VhostIOVATree - * - * Store and search IOVA -> Translated mappings and the reverse, from - * translated address to IOVA. + * VhostIOVATree, able to: + * - Translate iova address + * - Reverse translate iova address (from translated to iova) + * - Allocate IOVA regions for translated range (potentially slow operatio= n) * * Note that it cannot remove nodes. */ @@ -54,5 +55,7 @@ const VhostDMAMap *vhost_iova_tree_find_taddr(const Vhost= IOVATree *iova_rm, const VhostDMAMap *map); VhostDMAMapNewRC vhost_iova_tree_insert(VhostIOVATree *iova_rm, VhostDMAMap *map); +VhostDMAMapNewRC vhost_iova_tree_alloc(VhostIOVATree *iova_rm, + VhostDMAMap *map); =20 #endif diff --git a/hw/virtio/vhost-iova-tree.c b/hw/virtio/vhost-iova-tree.c index 2900390a1e..7699d96bbb 100644 --- a/hw/virtio/vhost-iova-tree.c +++ b/hw/virtio/vhost-iova-tree.c @@ -187,8 +187,30 @@ const VhostDMAMap *vhost_iova_tree_find_taddr(const Vh= ostIOVATree *tree, vhost_iova_tree_cmp_iova); } =20 +static bool vhost_iova_tree_find_iova_hole(const GArray *iova_map, + const VhostDMAMap *map, + const VhostDMAMap **prev_elem) +{ + size_t i; + hwaddr iova =3D 0; + + *prev_elem =3D NULL; + for (i =3D 0; i < iova_map->len; i++) { + const VhostDMAMap *next =3D &g_array_index(iova_map, typeof(*next)= , i); + hwaddr hole_end =3D next->iova; + if (map->size < hole_end - iova) { + return true; + } + + iova =3D next->iova + next->size + 1; + *prev_elem =3D next; + } + + return ((hwaddr)-1 - iova) > iova_map->len; +} + /** - * Insert a new map + * Insert a new map - internal * * @tree The iova tree * @map The iova map @@ -197,10 +219,13 @@ const VhostDMAMap *vhost_iova_tree_find_taddr(const V= hostIOVATree *tree, * - VHOST_DMA_MAP_OK if the map fits in the container * - VHOST_DMA_MAP_INVALID if the map does not make sense (like size overf= low) * - VHOST_DMA_MAP_OVERLAP if the tree already contains that map - * Can query the assignated iova in map. + * - VHOST_DMA_MAP_NO_SPACE if iova_rm cannot allocate more space. + * + * It returns assignated iova in map->iova if return value is VHOST_DMA_MA= P_OK. */ -VhostDMAMapNewRC vhost_iova_tree_insert(VhostIOVATree *tree, - VhostDMAMap *map) +static VhostDMAMapNewRC vhost_iova_tree_insert_int(VhostIOVATree *tree, + VhostDMAMap *map, + bool allocate) { const VhostDMAMap *qemu_prev, *iova_prev; int find_prev_rc; @@ -210,12 +235,27 @@ VhostDMAMapNewRC vhost_iova_tree_insert(VhostIOVATree= *tree, return VHOST_DMA_MAP_INVALID; } =20 - /* Check for duplicates, and save position for insertion */ - find_prev_rc =3D vhost_iova_tree_find_prev(tree->iova_taddr_map, - vhost_iova_tree_cmp_iova, map, - &iova_prev); - if (find_prev_rc =3D=3D VHOST_DMA_MAP_OVERLAP) { - return VHOST_DMA_MAP_OVERLAP; + if (allocate) { + /* Search for a hole in iova space big enough */ + bool fit =3D vhost_iova_tree_find_iova_hole(tree->iova_taddr_map, = map, + &iova_prev); + if (!fit) { + return VHOST_DMA_MAP_NO_SPACE; + } + + map->iova =3D iova_prev ? (iova_prev->iova + iova_prev->size) + 1 = : 0; + } else { + if (map->iova + map->size < map->iova) { + return VHOST_DMA_MAP_INVALID; + } + + /* Check for duplicates, and save position for insertion */ + find_prev_rc =3D vhost_iova_tree_find_prev(tree->iova_taddr_map, + vhost_iova_tree_cmp_iova,= map, + &iova_prev); + if (find_prev_rc =3D=3D VHOST_DMA_MAP_OVERLAP) { + return VHOST_DMA_MAP_OVERLAP; + } } =20 find_prev_rc =3D vhost_iova_tree_find_prev(tree->taddr_iova_map, @@ -229,3 +269,15 @@ VhostDMAMapNewRC vhost_iova_tree_insert(VhostIOVATree = *tree, vhost_iova_tree_insert_after(tree->taddr_iova_map, qemu_prev, map); return VHOST_DMA_MAP_OK; } + +VhostDMAMapNewRC vhost_iova_tree_insert(VhostIOVATree *tree, + VhostDMAMap *map) +{ + return vhost_iova_tree_insert_int(tree, map, false); +} + +VhostDMAMapNewRC vhost_iova_tree_alloc(VhostIOVATree *tree, + VhostDMAMap *map) +{ + return vhost_iova_tree_insert_int(tree, map, true); +} --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621443378; cv=none; d=zohomail.com; s=zohoarc; b=lCfsOgbBV8js2XaOj61GdwbjhaNUko4uaLJj5aYlcj5emJTywFksKc/qbion5A4MfQXqoXsRJGIGuNO3PfRBGs61wEZut52mDXoX0QsverRBlvV4OZOKQS2D2qneuuQ+3KEHEEiwXMJPulQ940aidydZTlU8QT7Exr6jM25JEDw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621443378; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=BdAE2rBCb1J3GMjsGYBKj9vUic0GjCsYOg0K5/xW1YA=; b=ZaSD+T1Pvgo7NU8nJreYLdHzTpEBa1OddDVa5hNGthOlcOqgMX6S6ht6gNKqvwfEiOVvSHwwjHrC3iKJBb6ODSP+QRYmapVLvWJ86wdg58EjDOK8/hHZRXOluyilt6n8DsNPH2qn/fiSUL7TKXqAe6W0V/cujps1pMrQAvGbUJA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621443378795768.1938239009687; Wed, 19 May 2021 09:56:18 -0700 (PDT) Received: from localhost ([::1]:33678 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPUW-00059R-E3 for importer@patchew.org; Wed, 19 May 2021 12:56:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35408) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7Q-0007GI-VK for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44239) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7O-00069J-Gx for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:24 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-420-dJSi6Z_qPWi3VxgnVkqmNA-1; Wed, 19 May 2021 12:32:18 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 639C3803621; Wed, 19 May 2021 16:32:13 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 698D95D6AC; Wed, 19 May 2021 16:32:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441941; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BdAE2rBCb1J3GMjsGYBKj9vUic0GjCsYOg0K5/xW1YA=; b=M4YApVc4YCtuuW55GRZqtZ3AuqKle+DK7whBCOiqkyJP/yw+955mOuX6QlBjx0gWUo5opr lisUAqcf1VyaJPTNplmUyUMSUtqwigHYFAIIXmps8b8onGtc/tMcplFNHia1oW6WDhi1vQ oYVxh5DUVRYOb0nhXwgI0hO3kJhOK6k= X-MC-Unique: dJSi6Z_qPWi3VxgnVkqmNA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 25/29] vhost: Add custom IOTLB translations to SVQ Date: Wed, 19 May 2021 18:28:59 +0200 Message-Id: <20210519162903.1172366-26-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Use translations added in IOVAReverseMaps in SVQ if the vhost device does not support the mapping of the full qemu's virtual address space. In other cases, Shadow Virtqueue still uses the qemu's virtual address of the buffer pointed by the descriptor, which has been translated already by qemu's VirtQueue machinery. Now every element needs to store the previous address also, so VirtQueue can consume the elements properly. This adds a little overhead per VQ element, having to allocate more memory to stash them. As a possible optimization, this allocation could be avoided if the descriptor is not a chain but a single one, but this is left undone. Checking also for vhost_set_iotlb_callback to send used ring remapping. This is only needed for kernel, and would print an error in case of vhost devices with its own mapping (vdpa). This could change for other callback, like checking for vhost_force_iommu, enable_custom_iommu, or another. Another option could be to, at least, extract the check of "is map(used, writable) needed?" in another function. But at the moment just copy the check used in vhost_dev_start here. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 134 ++++++++++++++++++++++++++--- hw/virtio/vhost.c | 29 +++++-- 2 files changed, 145 insertions(+), 18 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 934d3bb27b..a92da979d1 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -10,12 +10,19 @@ #include "hw/virtio/vhost-shadow-virtqueue.h" #include "hw/virtio/vhost.h" #include "hw/virtio/virtio-access.h" +#include "hw/virtio/vhost-iova-tree.h" =20 #include "standard-headers/linux/vhost_types.h" =20 #include "qemu/error-report.h" #include "qemu/main-loop.h" =20 +typedef struct SVQElement { + VirtQueueElement elem; + void **in_sg_stash; + void **out_sg_stash; +} SVQElement; + /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { /* Shadow vring */ @@ -50,8 +57,11 @@ typedef struct VhostShadowVirtqueue { /* Virtio device */ VirtIODevice *vdev; =20 + /* IOVA mapping if used */ + VhostIOVATree *iova_map; + /* Map for returning guest's descriptors */ - VirtQueueElement **ring_id_maps; + SVQElement **ring_id_maps; =20 /* Next head to expose to device */ uint16_t avail_idx_shadow; @@ -88,6 +98,66 @@ static void vhost_shadow_vq_set_notification(VhostShadow= Virtqueue *svq, } } =20 +static void vhost_shadow_vq_stash_addr(void ***stash, const struct iovec *= iov, + size_t num) +{ + size_t i; + + if (num =3D=3D 0) { + return; + } + + *stash =3D g_new(void *, num); + for (i =3D 0; i < num; ++i) { + (*stash)[i] =3D iov[i].iov_base; + } +} + +static void vhost_shadow_vq_unstash_addr(void **stash, + struct iovec *iov, + size_t num) +{ + size_t i; + + if (num =3D=3D 0) { + return; + } + + for (i =3D 0; i < num; ++i) { + iov[i].iov_base =3D stash[i]; + } + g_free(stash); +} + +static void vhost_shadow_vq_translate_addr(const VhostShadowVirtqueue *svq, + struct iovec *iovec, size_t num) +{ + size_t i; + + for (i =3D 0; i < num; ++i) { + VhostDMAMap needle =3D { + .translated_addr =3D iovec[i].iov_base, + .size =3D iovec[i].iov_len, + }; + size_t off; + + const VhostDMAMap *map =3D vhost_iova_tree_find_iova(svq->iova_map, + &needle); + /* + * Map cannot be NULL since iova map contains all guest space and + * qemu already has a physical address mapped + */ + assert(map); + + /* + * Map->iova chunk size is ignored. What to do if descriptor + * (addr, size) does not fit is delegated to the device. + */ + off =3D needle.translated_addr - map->translated_addr; + iovec[i].iov_base =3D (void *)(map->iova + off); + } +} + static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, const struct iovec *iovec, size_t num, bool more_descs, bool writ= e) @@ -118,8 +188,9 @@ static void vhost_vring_write_descs(VhostShadowVirtqueu= e *svq, } =20 static unsigned vhost_shadow_vq_add_split(VhostShadowVirtqueue *svq, - VirtQueueElement *elem) + SVQElement *svq_elem) { + VirtQueueElement *elem =3D &svq_elem->elem; int head; unsigned avail_idx; vring_avail_t *avail =3D svq->vring.avail; @@ -129,6 +200,16 @@ static unsigned vhost_shadow_vq_add_split(VhostShadowV= irtqueue *svq, /* We need some descriptors here */ assert(elem->out_num || elem->in_num); =20 + if (svq->iova_map) { + vhost_shadow_vq_stash_addr(&svq_elem->in_sg_stash, elem->in_sg, + elem->in_num); + vhost_shadow_vq_stash_addr(&svq_elem->out_sg_stash, elem->out_sg, + elem->out_num); + + vhost_shadow_vq_translate_addr(svq, elem->in_sg, elem->in_num); + vhost_shadow_vq_translate_addr(svq, elem->out_sg, elem->out_num); + } + vhost_vring_write_descs(svq, elem->out_sg, elem->out_num, elem->in_num > 0, false); vhost_vring_write_descs(svq, elem->in_sg, elem->in_num, false, true); @@ -150,7 +231,7 @@ static unsigned vhost_shadow_vq_add_split(VhostShadowVi= rtqueue *svq, } =20 static void vhost_shadow_vq_add(VhostShadowVirtqueue *svq, - VirtQueueElement *elem) + SVQElement *elem) { unsigned qemu_head =3D vhost_shadow_vq_add_split(svq, elem); =20 @@ -184,7 +265,7 @@ static void vhost_handle_guest_kick(EventNotifier *n) } =20 while (true) { - VirtQueueElement *elem =3D virtqueue_pop(svq->vq, sizeof(*elem= )); + SVQElement *elem =3D virtqueue_pop(svq->vq, sizeof(*elem)); if (!elem) { break; } @@ -210,7 +291,7 @@ static bool vhost_shadow_vq_more_used(VhostShadowVirtqu= eue *svq) return svq->used_idx !=3D svq->shadow_used_idx; } =20 -static VirtQueueElement *vhost_shadow_vq_get_buf(VhostShadowVirtqueue *svq) +static SVQElement *vhost_shadow_vq_get_buf(VhostShadowVirtqueue *svq) { vring_desc_t *descs =3D svq->vring.desc; const vring_used_t *used =3D svq->vring.used; @@ -235,7 +316,7 @@ static VirtQueueElement *vhost_shadow_vq_get_buf(VhostS= hadowVirtqueue *svq) svq->free_head =3D used_elem.id; =20 svq->used_idx++; - svq->ring_id_maps[used_elem.id]->len =3D used_elem.len; + svq->ring_id_maps[used_elem.id]->elem.len =3D used_elem.len; return g_steal_pointer(&svq->ring_id_maps[used_elem.id]); } =20 @@ -255,12 +336,21 @@ static void vhost_shadow_vq_handle_call_no_test(Event= Notifier *n) =20 vhost_shadow_vq_set_notification(svq, false); while (true) { - g_autofree VirtQueueElement *elem =3D vhost_shadow_vq_get_buf(= svq); - if (!elem) { + g_autofree SVQElement *svq_elem =3D vhost_shadow_vq_get_buf(sv= q); + VirtQueueElement *elem; + if (!svq_elem) { break; } =20 assert(i < svq->vring.num); + elem =3D &svq_elem->elem; + + if (svq->iova_map) { + vhost_shadow_vq_unstash_addr(svq_elem->in_sg_stash, + elem->in_sg, elem->in_num); + vhost_shadow_vq_unstash_addr(svq_elem->out_sg_stash, + elem->out_sg, elem->out_num); + } virtqueue_fill(vq, elem, elem->len, i++); } =20 @@ -455,14 +545,27 @@ void vhost_shadow_vq_stop(struct vhost_dev *dev, =20 =20 for (i =3D 0; i < svq->vring.num; ++i) { - g_autofree VirtQueueElement *elem =3D svq->ring_id_maps[i]; + g_autofree SVQElement *svq_elem =3D svq->ring_id_maps[i]; + VirtQueueElement *elem; + + if (!svq_elem) { + continue; + } + + elem =3D &svq_elem->elem; + + if (svq->iova_map) { + vhost_shadow_vq_unstash_addr(svq_elem->in_sg_stash, elem->in_s= g, + elem->in_num); + vhost_shadow_vq_unstash_addr(svq_elem->out_sg_stash, elem->out= _sg, + elem->out_num); + } + /* * Although the doc says we must unpop in order, it's ok to unpop * everything. */ - if (elem) { - virtqueue_unpop(svq->vq, elem, elem->len); - } + virtqueue_unpop(svq->vq, elem, elem->len); } } =20 @@ -504,11 +607,16 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhos= t_dev *dev, int idx) memset(svq->vring.desc, 0, driver_size); svq->vring.used =3D qemu_memalign(qemu_real_host_page_size, device_siz= e); memset(svq->vring.used, 0, device_size); + + if (vhost_has_limited_iova_range(dev)) { + svq->iova_map =3D &dev->iova_map; + } + for (i =3D 0; i < num - 1; i++) { svq->vring.desc[i].next =3D cpu_to_le16(i + 1); } =20 - svq->ring_id_maps =3D g_new0(VirtQueueElement *, num); + svq->ring_id_maps =3D g_new0(SVQElement *, num); event_notifier_set_handler(&svq->call_notifier, vhost_shadow_vq_handle_call); return g_steal_pointer(&svq); diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 925d2146a4..4339b899ea 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1306,7 +1306,13 @@ static bool vhost_shadow_vq_start_store_sections(str= uct vhost_dev *dev) .perm =3D VHOST_ACCESS_RW, }; =20 - VhostDMAMapNewRC r =3D vhost_iova_tree_insert(&dev->iova_map, ®= ion); + VhostDMAMapNewRC r; + + if (vhost_has_limited_iova_range(dev)) { + r =3D vhost_iova_tree_alloc(&dev->iova_map, ®ion); + } else { + r =3D vhost_iova_tree_insert(&dev->iova_map, ®ion); + } assert(r =3D=3D VHOST_DMA_MAP_OK); } =20 @@ -1351,11 +1357,24 @@ static bool vhost_sw_live_migration_start_vq(struct= vhost_dev *dev, .perm =3D VHOST_ACCESS_RW, }; =20 - r =3D vhost_iova_tree_insert(&dev->iova_map, &driver_region); - assert(r =3D=3D VHOST_DMA_MAP_OK); + if (vhost_has_limited_iova_range(dev)) { + r =3D vhost_iova_tree_alloc(&dev->iova_map, &driver_region); + assert(r =3D=3D VHOST_DMA_MAP_OK); + + r =3D vhost_iova_tree_alloc(&dev->iova_map, &device_region); + assert(r =3D=3D VHOST_DMA_MAP_OK); =20 - r =3D vhost_iova_tree_insert(&dev->iova_map, &device_region); - assert(r =3D=3D VHOST_DMA_MAP_OK); + addr.avail_user_addr =3D driver_region.iova + addr.avail_user_addr + - addr.desc_user_addr; + addr.desc_user_addr =3D driver_region.iova; + addr.used_user_addr =3D device_region.iova; + } else { + r =3D vhost_iova_tree_insert(&dev->iova_map, &driver_region); + assert(r =3D=3D VHOST_DMA_MAP_OK); + + r =3D vhost_iova_tree_insert(&dev->iova_map, &device_region); + assert(r =3D=3D VHOST_DMA_MAP_OK); + } =20 vhost_virtqueue_stop(dev, dev->vdev, &dev->vqs[idx], dev->vq_index + i= dx); ok =3D vhost_shadow_vq_start(dev, idx, dev->shadow_vqs[idx]); --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621443299; cv=none; d=zohomail.com; s=zohoarc; b=gLiAcz2gdR8PeB3m0guoKro0TZKhubAocT0/DZ4CdKOsgcm3Ndb2FusBVenXDbYAnZq/idaCCar2cuFqMrdQ8hsPMq9hJyE3vZorsax1jnZ4kUYJN4Ruy/SIU4TadRe/H51QHAy3xxrtap/J2gohRQkol8df6r7TgSdeoWL7HjU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621443299; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=1i1NEzjfpXbKffh1W/jtWwyo2lcMMIpkTS0taNvfQeE=; b=TI7S1W9f8WfFzAeuQoVVzWDefj8eP+f14Z2t2nGwu7YKrd4Q/1qGUoYvv5DeFVgb8YMfsxnL0OO6JVaLm/Cue2rEJe9GNi+lOBaJ6IZbzGeo1rmcq2O+ZSMV45eDci0XgSwRJypUrPLxwNlw9VEp9moaHL82L19O2qFTZ8rV+as= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621443299627117.62894751478905; Wed, 19 May 2021 09:54:59 -0700 (PDT) Received: from localhost ([::1]:57612 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPTG-0002Gb-9V for importer@patchew.org; Wed, 19 May 2021 12:54:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35424) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7T-0007Ki-Nt for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:28 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:39872) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7Q-0006AC-UZ for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:27 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-193-74_U5WOZOrqi5la8NF6WbA-1; Wed, 19 May 2021 12:32:22 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4BE64107ACCA; Wed, 19 May 2021 16:32:21 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id BCC225D6AC; Wed, 19 May 2021 16:32:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441944; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1i1NEzjfpXbKffh1W/jtWwyo2lcMMIpkTS0taNvfQeE=; b=TAWpo8zbX0QN16fp+NN1KBIWjrFecDDTHg9mRgI9i81rQkrihKX9ZyADbMCc4FhTu1Rzcm 594IfzLFIJTJyYSAKEvmElh79NT9/3xroS7gLRRxp24l/w6ZjOc0bVNlLnBcuwTq0YwRBP yIh593zU6L0sHLNv+jtkMXN8XHjRGpk= X-MC-Unique: 74_U5WOZOrqi5la8NF6WbA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 26/29] vhost: Map in vdpa-dev Date: Wed, 19 May 2021 18:29:00 +0200 Message-Id: <20210519162903.1172366-27-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Use and export vhost-vpda functions directly. In the final version, these methods needs to be exposed through VhostOps, or vhost-vdpa backend needs to be adapted to work with vhost_send_device_iotlb_msg in case its custom iommu is disabled. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-backend.h | 4 ++++ hw/virtio/vhost-vdpa.c | 2 +- hw/virtio/vhost.c | 18 ++++++++++++++++++ 3 files changed, 23 insertions(+), 1 deletion(-) diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-ba= ckend.h index f8eed2ace5..9d88074e4d 100644 --- a/include/hw/virtio/vhost-backend.h +++ b/include/hw/virtio/vhost-backend.h @@ -203,4 +203,8 @@ int vhost_backend_handle_iotlb_msg(struct vhost_dev *de= v, =20 int vhost_user_gpu_set_socket(struct vhost_dev *dev, int fd); =20 +struct vhost_vdpa; +int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr size, + void *vaddr, bool readonly); + #endif /* VHOST_BACKEND_H */ diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 9e7a0ce5e0..c742e6944e 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -36,7 +36,7 @@ static bool vhost_vdpa_listener_skipped_section(MemoryReg= ionSection *section) section->offset_within_address_space & (1ULL << 63); } =20 -static int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr si= ze, +int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr size, void *vaddr, bool readonly) { struct vhost_msg_v2 msg =3D {}; diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 4339b899ea..286863ad42 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1314,9 +1314,19 @@ static bool vhost_shadow_vq_start_store_sections(str= uct vhost_dev *dev) r =3D vhost_iova_tree_insert(&dev->iova_map, ®ion); } assert(r =3D=3D VHOST_DMA_MAP_OK); + r =3D vhost_vdpa_dma_map(dev->opaque, region.iova, region_size, + (void *)dev->mem->regions[idx].userspace_ad= dr, + false); + if (r !=3D 0) { + goto fail; + } } =20 return true; + +fail: + assert(0); + return false; } =20 /* @@ -1377,6 +1387,14 @@ static bool vhost_sw_live_migration_start_vq(struct = vhost_dev *dev, } =20 vhost_virtqueue_stop(dev, dev->vdev, &dev->vqs[idx], dev->vq_index + i= dx); + /* TODO: Why cannot make this read only? */ + r =3D vhost_vdpa_dma_map(dev->opaque, addr.desc_user_addr, driver_regi= on.size, + (void *)driver_region.translated_addr, false); + assert(r =3D=3D 0); + r =3D vhost_vdpa_dma_map(dev->opaque, addr.used_user_addr, device_regi= on.size, + (void *)device_region.translated_addr, false); + assert(r =3D=3D 0); + ok =3D vhost_shadow_vq_start(dev, idx, dev->shadow_vqs[idx]); if (unlikely(!ok)) { return false; --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621443511; cv=none; d=zohomail.com; s=zohoarc; b=gp+6ax1U/fTfWnIk1K9QpPo1KOmpSutWLQ0mefEo7Uyhfdrts6qZVrn0aGlcyDeqdKkDFrFt+YleDYJmCygY1zBymYb78XjAPsvUuNphufXHcjjO6paFqCX7tT+WArCZXUCRkiHxXtSO9+w9ge2OOFsfO0R2ogAJbZ+wSS1+kIU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621443511; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=MmSHVdF6N2KhRwS4pQZkGRb68Ei7kpvUrSWw6KsIbSo=; b=Kpv7c6wcr4F7dh/eicyI02OvatOINWGWvBOO9aKrgKUa3W+pGIYsBpphHWUhdv8DWFECV8dBm69NALNjExHLTxjg8FTjUeUb0ixGtBXhFhZpTzU5xwD2oh04TKtcUdXwty3YdyKpf+8W16GNcFPC4AoDdM+U05ZFXJSK//gFYEs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621443511377390.1485114993684; Wed, 19 May 2021 09:58:31 -0700 (PDT) Received: from localhost ([::1]:43870 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPWg-0003c4-DB for importer@patchew.org; Wed, 19 May 2021 12:58:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35518) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7m-0007dR-KZ for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:23326) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7l-0006GN-2t for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:46 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-568-gHdCR_5UOKesYYDvKmCCfw-1; Wed, 19 May 2021 12:32:41 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0EEE71854E21; Wed, 19 May 2021 16:32:40 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id A0D825D6AC; Wed, 19 May 2021 16:32:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441964; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MmSHVdF6N2KhRwS4pQZkGRb68Ei7kpvUrSWw6KsIbSo=; b=UinSAH3JVzouvpL7C4OQ0WN7t1o7wK131txcP7XQl+SLDSL8V1rsvclxO1sYmCv4+G0x3R JxBLLpPbfqz7MMffoz0DMW0d8sLOWWiivzUvSEL5w6QzuJAwV+59YDB2L2Q5YQYoQFwjyX nHueCvUdFPFdZ1s1dZd6xU0r44tpeH8= X-MC-Unique: gHdCR_5UOKesYYDvKmCCfw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 27/29] vhost-vdpa: Implement vhost_vdpa_vring_pause operation Date: Wed, 19 May 2021 18:29:01 +0200 Message-Id: <20210519162903.1172366-28-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This uses the status bit DEVICE_STOPPED, that is currently discussed in VirtIO, and is implemented in qemu VirtIO-net devices in previous commits. Removal of _S_DEVICE_STOPPED can be done in the future if an use case arises. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-vdpa.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index c742e6944e..dfb465be96 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -618,6 +618,19 @@ static int vhost_vdpa_get_iova_range(struct vhost_dev = *dev, return ret; } =20 +static int vhost_vdpa_vring_pause(struct vhost_dev *dev) +{ + int r; + uint8_t status; + + vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DEVICE_STOPPED); + do { + r =3D vhost_vdpa_call(dev, VHOST_VDPA_GET_STATUS, &status); + } while (r =3D=3D 0 && !(status & VIRTIO_CONFIG_S_DEVICE_STOPPED)); + + return 0; +} + const VhostOps vdpa_ops =3D { .backend_type =3D VHOST_BACKEND_TYPE_VDPA, .vhost_backend_init =3D vhost_vdpa_init, @@ -650,6 +663,7 @@ const VhostOps vdpa_ops =3D { .vhost_get_device_id =3D vhost_vdpa_get_device_id, .vhost_vq_get_addr =3D vhost_vdpa_vq_get_addr, .vhost_force_iommu =3D vhost_vdpa_force_iommu, + .vhost_vring_pause =3D vhost_vdpa_vring_pause, .vhost_enable_custom_iommu =3D vhost_vdpa_enable_custom_iommu, .vhost_get_iova_range =3D vhost_vdpa_get_iova_range, }; --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621443704; cv=none; d=zohomail.com; s=zohoarc; b=G+ZZx5VEy/79ePYM8X7tGYzuRHdWzjMJy7UabKlXje0uMJe5O4ziO+51615gXwvMWw/j+NuoJbnULkpW7Cm5g5Q94RPyj/JQNmWV2p9s5I9et5Qbe6QlRfgbYhjpWVj1mh7aezpaY2gYj/ICYgYmkSYGndmSC34dwSgkOlGhl/o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621443704; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=+g8d0jKeRijJXiCoq8EJzNGXAKyOpUZngFpIrBOn7os=; b=iPGIi9N1+ZRhveQ+r1l75kE2/gDZwhyEL6xuFDYj3ZaLLKbLuzNy1hA25G5irHb2Grn7TDbVhsFaT98WrUhIRa3qEX3+1ERLaw7RmelUULvRN/Rleigi7+Dt2IWVR2euAg/R/pDQdzCwUMEhikE6voVOWTAMhqHieTjzCZ5UL4w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621443704053889.7264248110979; Wed, 19 May 2021 10:01:44 -0700 (PDT) Received: from localhost ([::1]:52424 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPZj-0000zF-Id for importer@patchew.org; Wed, 19 May 2021 13:01:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35544) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7r-0007fg-QL for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:21460) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7m-0006Gw-SF for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:51 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-239-5x9-pw_HObC5X-ENtfFT-A-1; Wed, 19 May 2021 12:32:44 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4F5EA101962A; Wed, 19 May 2021 16:32:43 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 62DFE5D6AC; Wed, 19 May 2021 16:32:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441966; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+g8d0jKeRijJXiCoq8EJzNGXAKyOpUZngFpIrBOn7os=; b=REuiomhS8ymmMn7XkdeFFVxDZ/SR8sCCIRKd9pIFfqHz4xGidhxveBoI9pppYbmyLLuK86 JwI7Vy6h79OX88Mq9GtFtKZQpQDXXmUK8s2PB4YB3/nWUpFWTx1SvHR6dQtykhzQvIJy1P N2WwVmy6C2Me+QdmwkVPhf5UaoXmVDA= X-MC-Unique: 5x9-pw_HObC5X-ENtfFT-A-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 28/29] vhost-vdpa: never map with vDPA listener Date: Wed, 19 May 2021 18:29:02 +0200 Message-Id: <20210519162903.1172366-29-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This commit is a workaround that will not go to the final version. vp_vdpa is not able to reset all IOTLBs, so we force to not to map them in the first place. Checkpath detects a few errors because of #if 0 / #endif pairs, but it's the less intrusive way to comment out all the code we want to skip. Since this commit is not intended to go to the final series, I left it that way. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-vdpa.h | 2 +- hw/virtio/vhost-vdpa.c | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 9b81a409da..06afe42ab6 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -17,7 +17,7 @@ typedef struct vhost_vdpa { int device_fd; uint32_t msg_type; - MemoryListener listener; + /* MemoryListener listener; */ struct vhost_dev *dev; } VhostVDPA; =20 diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index dfb465be96..30e4e306fb 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -23,6 +23,7 @@ #include "trace.h" #include "qemu-common.h" =20 +#if 0 static bool vhost_vdpa_listener_skipped_section(MemoryRegionSection *secti= on) { return (!memory_region_is_ram(section->mr) && @@ -35,6 +36,7 @@ static bool vhost_vdpa_listener_skipped_section(MemoryReg= ionSection *section) */ section->offset_within_address_space & (1ULL << 63); } +#endif =20 int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova, hwaddr size, void *vaddr, bool readonly) @@ -62,6 +64,7 @@ int vhost_vdpa_dma_map(struct vhost_vdpa *v, hwaddr iova,= hwaddr size, return ret; } =20 +#if 0 static int vhost_vdpa_dma_unmap(struct vhost_vdpa *v, hwaddr iova, hwaddr size) { @@ -246,6 +249,7 @@ static const MemoryListener vhost_vdpa_memory_listener = =3D { .region_add =3D vhost_vdpa_listener_region_add, .region_del =3D vhost_vdpa_listener_region_del, }; +#endif =20 static int vhost_vdpa_call(struct vhost_dev *dev, unsigned long int reques= t, void *arg) @@ -274,6 +278,7 @@ static void vhost_vdpa_add_status(struct vhost_dev *dev= , uint8_t status) =20 static int vhost_vdpa_enable_custom_iommu(struct vhost_dev *dev, bool enab= le) { +#if 0 struct vhost_vdpa *v =3D dev->opaque; hwaddr iova_range_last =3D dev->iova_range.last; if (iova_range_last !=3D (hwaddr)-1) { @@ -291,6 +296,7 @@ static int vhost_vdpa_enable_custom_iommu(struct vhost_= dev *dev, bool enable) memory_listener_unregister(&v->listener); return vhost_vdpa_dma_unmap(v, dev->iova_range.first, iova_range_l= ast); } +#endif =20 return 0; } @@ -307,7 +313,7 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void = *opaque) dev->opaque =3D opaque ; vhost_vdpa_call(dev, VHOST_GET_FEATURES, &features); dev->backend_features =3D features; - v->listener =3D vhost_vdpa_memory_listener; + /* v->listener =3D vhost_vdpa_memory_listener; */ v->msg_type =3D VHOST_IOTLB_MSG_V2; =20 vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | --=20 2.27.0 From nobody Wed May 15 16:32:08 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1621442914; cv=none; d=zohomail.com; s=zohoarc; b=JNIeR7mRYs3t08Fhgein1YP44QThZ3Y76mLGIoU+4at4ClCcVZbS5MqniGwO2TmtWyCpkcb9VOKyBKCNRXcnKKHiJMYYl0KuivOubd29PvaVj2l0g3vDIwfem1gOSkiSgoSsoo7QxV5DOBpzjXiXYxTEpro8/txOo2p3UeTyPD8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1621442914; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Va3lQ1jFZTAO+vX7RJUj5VEK4CtKAhX+9xJSzKnaFW8=; b=YvIvvLXbuct0mGY4ErRd2eRVahCSzlfKs97BGsbLSBdQwtHVVN0PxC/jgCE0QsIJOywP8c5OMWqOq/Ig4dc/Ik0YJBPeBmP6XNsR5bz8TUDAlT0H3u41ftUZDgxMEPfbeu0OgWPMOZpevf4WqyyGAk6ZYheKSNDhsWrf173153Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1621442914735496.61106584145; Wed, 19 May 2021 09:48:34 -0700 (PDT) Received: from localhost ([::1]:37686 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ljPN3-0005RN-Sq for importer@patchew.org; Wed, 19 May 2021 12:48:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35604) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7y-0007hc-4e for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46390) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ljP7u-0006Ip-E3 for qemu-devel@nongnu.org; Wed, 19 May 2021 12:32:57 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-105-LriYgRduOnyrXhT_r14bpw-1; Wed, 19 May 2021 12:32:51 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0A5AB1854E2D; Wed, 19 May 2021 16:32:50 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-65.ams2.redhat.com [10.36.113.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id A1A225D6AC; Wed, 19 May 2021 16:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1621441973; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Va3lQ1jFZTAO+vX7RJUj5VEK4CtKAhX+9xJSzKnaFW8=; b=bz9Ezg1ay0i8t4whjN81mRUchum0VhuIsUTWMnVPMlJ4LYrQ57JNj62Qw/z+eXoYnnhbGl NGGjBl6wq0F0vxKNFsUj9wzSNsyQEDJi2kgWDS4+zRTcIzJYA3iRSoLpDUd79QlXj8tifn kM7qS7kgEQWrp5Y1jhMvyEFQ69abtyE= X-MC-Unique: LriYgRduOnyrXhT_r14bpw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v3 29/29] vhost: Start vhost-vdpa SVQ directly Date: Wed, 19 May 2021 18:29:03 +0200 Message-Id: <20210519162903.1172366-30-eperezma@redhat.com> In-Reply-To: <20210519162903.1172366-1-eperezma@redhat.com> References: <20210519162903.1172366-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.39, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Stefan Hajnoczi , Eli Cohen , Michael Lilja , Stefano Garzarella Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Since it does not have sense to keep a non-working vdpa device, start directly in SVQ mode. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 286863ad42..fd812e1a80 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1386,7 +1386,6 @@ static bool vhost_sw_live_migration_start_vq(struct v= host_dev *dev, assert(r =3D=3D VHOST_DMA_MAP_OK); } =20 - vhost_virtqueue_stop(dev, dev->vdev, &dev->vqs[idx], dev->vq_index + i= dx); /* TODO: Why cannot make this read only? */ r =3D vhost_vdpa_dma_map(dev->opaque, addr.desc_user_addr, driver_regi= on.size, (void *)driver_region.translated_addr, false); @@ -1467,6 +1466,11 @@ static int vhost_sw_live_migration_start(struct vhos= t_dev *dev) =20 /* Can be read by vhost_virtqueue_mask, from vm exit */ dev->shadow_vqs_enabled =3D true; + + /* Reset device, so SVQ can assign its address */ + r =3D dev->vhost_ops->vhost_dev_start(dev, false); + assert(r =3D=3D 0); + for (idx =3D 0; idx < dev->nvqs; ++idx) { bool ok =3D vhost_sw_live_migration_start_vq(dev, idx); if (unlikely(!ok)) { @@ -2107,6 +2111,8 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODev= ice *vdev) vhost_device_iotlb_miss(hdev, vq->used_phys, true); } } + + vhost_sw_live_migration_start(hdev); return 0; fail_log: vhost_log_put(hdev, false); --=20 2.27.0