From nobody Sat Feb 7 03:55:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1622623345; cv=none; d=zohomail.com; s=zohoarc; b=TTM7K0Cl5GOaOJpUb4fuXOYZMOjLmdFxVxuE8PCh6X71bDLGnPv5RV7jwZfIAuVVxGKCz2GkpMCITQucPs2kLgCOK9X66/Tgl/dLL6EyM6uuQgVk6UAqpQWUXdg1MqWzVU/HWNObGcu1eWJUcMTUyJAqEy8D632S2W+HwfG+rTA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622623345; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=eAYC3tJKs+qL04BDfVKy+xILBP58PJDFLvNSXkSRuJQ=; b=dEC9KdC32515aQz+7727q81EJxK3xgDcQg5UCOzioP/L6RsfHwiFg9sd6SRB2G6f4zX5t8EkYWB/xf8v+N2jnOa02Ospa+1gj33T6FfCA9BDnpN35CBA7S5sqLV0TlDjkqMiboVbD0I2evpxIDvKzK/K7xzpMUGa236hGYl75Ns= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1622623345235947.0212580268441; Wed, 2 Jun 2021 01:42:25 -0700 (PDT) Received: from localhost ([::1]:53706 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1loMSG-00080K-5I for importer@patchew.org; Wed, 02 Jun 2021 04:42:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48688) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1loMRL-0006NR-2H for qemu-devel@nongnu.org; Wed, 02 Jun 2021 04:41:27 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44704) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1loMRJ-0003fm-HY for qemu-devel@nongnu.org; Wed, 02 Jun 2021 04:41:26 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-412-fTy7F-fkP7iIJ-IghieAoQ-1; Wed, 02 Jun 2021 04:41:23 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5090F8042B7; Wed, 2 Jun 2021 08:41:22 +0000 (UTC) Received: from localhost.localdomain (ovpn-12-208.pek2.redhat.com [10.72.12.208]) by smtp.corp.redhat.com (Postfix) with ESMTP id 103E85C6DE; Wed, 2 Jun 2021 08:41:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622623284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eAYC3tJKs+qL04BDfVKy+xILBP58PJDFLvNSXkSRuJQ=; b=LDZmtAg+72yTNzPi6WC+SyVR6PeVofThk/GQWpY9hsJbJ3MQMRuN7hdPY2mN1ejQjvF/oH qOAF3Ij/0Ezze1rdVZuk+u+OSO7KkWHD7yCngERdnZ8gEvpMaREK1wRwnJmY93Pm2bBe/9 NrhjbnBdZa1nV+4S5E5ZAE4RrPS9oBI= X-MC-Unique: fTy7F-fkP7iIJ-IghieAoQ-1 From: Jason Wang To: mst@redhat.com, qemu-devel@nongnu.org Subject: [PATCH V2 1/2] vhost-vdpa: skip ram device from the IOTLB mapping Date: Wed, 2 Jun 2021 16:41:05 +0800 Message-Id: <20210602084106.43186-2-jasowang@redhat.com> In-Reply-To: <20210602084106.43186-1-jasowang@redhat.com> References: <20210602084106.43186-1-jasowang@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jasowang@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.371, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: si-wei.liu@oracle.com, elic@nvidia.com, lingshan.zhu@intel.com, Jason Wang Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" vDPA is not tie to any specific hardware, for safety and simplicity, vhost-vDPA doesn't allow MMIO area to be mapped via IOTLB. Only the doorbell could be mapped via mmap(). So this patch exclude skip the ram device from the IOTLB mapping. Signed-off-by: Jason Wang Reviewed-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 01d2101d09..dd4321bac2 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -27,6 +27,8 @@ static bool vhost_vdpa_listener_skipped_section(MemoryReg= ionSection *section) { return (!memory_region_is_ram(section->mr) && !memory_region_is_iommu(section->mr)) || + /* vhost-vDPA doesn't allow MMIO to be mapped */ + memory_region_is_ram_device(section->mr) || /* * Sizing an enabled 64-bit BAR can cause spurious mappings to * addresses in the upper part of the 64-bit address space. Th= ese @@ -171,22 +173,12 @@ static void vhost_vdpa_listener_region_add(MemoryList= ener *listener, vaddr, section->readonly); if (ret) { error_report("vhost vdpa map fail!"); - if (memory_region_is_ram_device(section->mr)) { - /* Allow unexpected mappings not to be fatal for RAM devices */ - error_report("map ram fail!"); - return ; - } goto fail; } =20 return; =20 fail: - if (memory_region_is_ram_device(section->mr)) { - error_report("failed to vdpa_dma_map. pci p2p may not work"); - return; - - } /* * On the initfn path, store the first error in the container so we * can gracefully fail. Runtime, there's not much we can do other --=20 2.25.1 From nobody Sat Feb 7 03:55:37 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1622623414; cv=none; d=zohomail.com; s=zohoarc; b=FNwNsV/CAklSd3h/gaSmqZdQe4dTrklPvLO3XuxlqC7q6Y1hgbiXFbe813bzpQbXE8acOaNUt2wDVHcxL+JKj9cEnclh6Vx1Dbe4rmYimN7/n9gfQcjXpXx9AMKnTivtKZ1fnvcLaOe7nzzdgVK9Gs3yFKFlC+6ILDXm7JPaLeU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622623414; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=EcCzrKSJCUwPbt76tLp4aCJ/tjnA4Nnzfj4IZ9JFgvQ=; b=eNo2qLl0NPqrbR2elrKfoo/bk91n0eJ6D+kPmOKKIDEJquvHnucbYgS0OV6PPx8TwAscjHdAr10ifUyK0W208bp+jr8HR4WjUq9NijZ7EazAR/CwJJ3iKgtxi0V7sVtQEVetHgaQRhpSZLcO7pHb0FyFAPhEz2vR7aahbkVXDcw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1622623414442207.17105536873567; Wed, 2 Jun 2021 01:43:34 -0700 (PDT) Received: from localhost ([::1]:56776 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1loMTN-0001cv-9E for importer@patchew.org; Wed, 02 Jun 2021 04:43:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1loMRO-0006XY-7D for qemu-devel@nongnu.org; Wed, 02 Jun 2021 04:41:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:24039) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1loMRM-0003hj-Ed for qemu-devel@nongnu.org; Wed, 02 Jun 2021 04:41:29 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-213-gGWxmnwOPaeGaElRrozrYQ-1; Wed, 02 Jun 2021 04:41:26 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E3D771009446; Wed, 2 Jun 2021 08:41:24 +0000 (UTC) Received: from localhost.localdomain (ovpn-12-208.pek2.redhat.com [10.72.12.208]) by smtp.corp.redhat.com (Postfix) with ESMTP id CCB115C6DE; Wed, 2 Jun 2021 08:41:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1622623287; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EcCzrKSJCUwPbt76tLp4aCJ/tjnA4Nnzfj4IZ9JFgvQ=; b=NPtC0OsIEcuZqSNDKVtjEpWe/Uko4NioHQqRUH7obrrSFOQo1ZP5TlNOTbDx/2tee6NlGO W7y3EIRJ/FnVrvcJbnwk2fALTZopVEmRKr45m6/6HxzVkJb1bAN60FRsH5PEnOGFlHaTzC ej97+d69o+sbbFqBxY4fKWDuGATsmME= X-MC-Unique: gGWxmnwOPaeGaElRrozrYQ-1 From: Jason Wang To: mst@redhat.com, qemu-devel@nongnu.org Subject: [PATCH V2 2/2] vhost-vdpa: map virtqueue notification area if possible Date: Wed, 2 Jun 2021 16:41:06 +0800 Message-Id: <20210602084106.43186-3-jasowang@redhat.com> In-Reply-To: <20210602084106.43186-1-jasowang@redhat.com> References: <20210602084106.43186-1-jasowang@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=jasowang@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.371, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: si-wei.liu@oracle.com, elic@nvidia.com, lingshan.zhu@intel.com, Jason Wang Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" This patch implements the vq notification mapping support for vhost-vDPA. This is simply done by using mmap()/munmap() for the vhost-vDPA fd during device start/stop. For the device without notification mapping support, we fall back to eventfd based notification gracefully. Signed-off-by: Jason Wang Reviewed-by: Si-Wei Liu --- Changes since v1: - use dev->vq_index to calculate the virtqueue index - remove the unused host_notifier_set --- hw/virtio/vhost-vdpa.c | 85 ++++++++++++++++++++++++++++++++++ include/hw/virtio/vhost-vdpa.h | 6 +++ 2 files changed, 91 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index dd4321bac2..f9a86afe64 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -285,12 +285,95 @@ static int vhost_vdpa_init(struct vhost_dev *dev, voi= d *opaque) return 0; } =20 +static void vhost_vdpa_host_notifier_uninit(struct vhost_dev *dev, + int queue_index) +{ + size_t page_size =3D qemu_real_host_page_size; + struct vhost_vdpa *v =3D dev->opaque; + VirtIODevice *vdev =3D dev->vdev; + VhostVDPAHostNotifier *n; + + n =3D &v->notifier[queue_index]; + + if (n->addr) { + virtio_queue_set_host_notifier_mr(vdev, queue_index, &n->mr, false= ); + object_unparent(OBJECT(&n->mr)); + munmap(n->addr, page_size); + n->addr =3D NULL; + } +} + +static void vhost_vdpa_host_notifiers_uninit(struct vhost_dev *dev, int n) +{ + int i; + + for (i =3D 0; i < n; i++) { + vhost_vdpa_host_notifier_uninit(dev, i); + } +} + +static int vhost_vdpa_host_notifier_init(struct vhost_dev *dev, int queue_= index) +{ + size_t page_size =3D qemu_real_host_page_size; + struct vhost_vdpa *v =3D dev->opaque; + VirtIODevice *vdev =3D dev->vdev; + VhostVDPAHostNotifier *n; + int fd =3D v->device_fd; + void *addr; + char *name; + + vhost_vdpa_host_notifier_uninit(dev, queue_index); + + n =3D &v->notifier[queue_index]; + + addr =3D mmap(NULL, page_size, PROT_WRITE, MAP_SHARED, fd, + queue_index * page_size); + if (addr =3D=3D MAP_FAILED) { + goto err; + } + + name =3D g_strdup_printf("vhost-vdpa/host-notifier@%p mmaps[%d]", + v, queue_index); + memory_region_init_ram_device_ptr(&n->mr, OBJECT(vdev), name, + page_size, addr); + g_free(name); + + if (virtio_queue_set_host_notifier_mr(vdev, queue_index, &n->mr, true)= ) { + munmap(addr, page_size); + goto err; + } + n->addr =3D addr; + + return 0; + +err: + return -1; +} + +static void vhost_vdpa_host_notifiers_init(struct vhost_dev *dev) +{ + int i; + + for (i =3D dev->vq_index; i < dev->vq_index + dev->nvqs; i++) { + if (vhost_vdpa_host_notifier_init(dev, i)) { + goto err; + } + } + + return; + +err: + vhost_vdpa_host_notifiers_uninit(dev, i); + return; +} + static int vhost_vdpa_cleanup(struct vhost_dev *dev) { struct vhost_vdpa *v; assert(dev->vhost_ops->backend_type =3D=3D VHOST_BACKEND_TYPE_VDPA); v =3D dev->opaque; trace_vhost_vdpa_cleanup(dev, v); + vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); memory_listener_unregister(&v->listener); =20 dev->opaque =3D NULL; @@ -467,6 +550,7 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev, = bool started) if (started) { uint8_t status =3D 0; memory_listener_register(&v->listener, &address_space_memory); + vhost_vdpa_host_notifiers_init(dev); vhost_vdpa_set_vring_ready(dev); vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK); vhost_vdpa_call(dev, VHOST_VDPA_GET_STATUS, &status); @@ -476,6 +560,7 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev, = bool started) vhost_vdpa_reset_device(dev); vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER); + vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); memory_listener_unregister(&v->listener); =20 return 0; diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 9b81a409da..56bef30ec2 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -14,11 +14,17 @@ =20 #include "hw/virtio/virtio.h" =20 +typedef struct VhostVDPAHostNotifier { + MemoryRegion mr; + void *addr; +} VhostVDPAHostNotifier; + typedef struct vhost_vdpa { int device_fd; uint32_t msg_type; MemoryListener listener; struct vhost_dev *dev; + VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX]; } VhostVDPA; =20 extern AddressSpace address_space_memory; --=20 2.25.1