From nobody Sun Feb 8 19:38:01 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zoho.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1492714156318707.7017459136453; Thu, 20 Apr 2017 11:49:16 -0700 (PDT) Received: from localhost ([::1]:55469 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d1H8k-0002Z1-Bb for importer@patchew.org; Thu, 20 Apr 2017 14:49:14 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46983) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d1H70-0001Sh-Iv for qemu-devel@nongnu.org; Thu, 20 Apr 2017 14:47:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d1H6w-0004MT-6N for qemu-devel@nongnu.org; Thu, 20 Apr 2017 14:47:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:41806) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d1H6v-0004MN-TB for qemu-devel@nongnu.org; Thu, 20 Apr 2017 14:47:22 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AFA458FD1D; Thu, 20 Apr 2017 18:47:20 +0000 (UTC) Received: from localhost (ovpn-116-28.gru2.redhat.com [10.97.116.28]) by smtp.corp.redhat.com (Postfix) with ESMTP id 804CB17AF5; Thu, 20 Apr 2017 18:47:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com AFA458FD1D Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx02.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ehabkost@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com AFA458FD1D From: Eduardo Habkost To: Peter Maydell , qemu-devel@nongnu.org Date: Thu, 20 Apr 2017 15:46:51 -0300 Message-Id: <20170420184705.25018-2-ehabkost@redhat.com> In-Reply-To: <20170420184705.25018-1-ehabkost@redhat.com> References: <20170420184705.25018-1-ehabkost@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Thu, 20 Apr 2017 18:47:20 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PULL 01/15] memory: add section range info for IOMMU notifier X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marcel Apfelbaum , Igor Mammedov , Peter Xu , "Michael S. Tsirkin" Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Peter Xu In this patch, IOMMUNotifier.{start|end} are introduced to store section information for a specific notifier. When notification occurs, we not only check the notification type (MAP|UNMAP), but also check whether the notified iova range overlaps with the range of specific IOMMU notifier, and skip those notifiers if not in the listened range. When removing an region, we need to make sure we removed the correct VFIOGuestIOMMU by checking the IOMMUNotifier.start address as well. This patch is solving the problem that vfio-pci devices receive duplicated UNMAP notification on x86 platform when vIOMMU is there. The issue is that x86 IOMMU has a (0, 2^64-1) IOMMU region, which is splitted by the (0xfee00000, 0xfeefffff) IRQ region. AFAIK this (splitted IOMMU region) is only happening on x86. This patch also helps vhost to leverage the new interface as well, so that vhost won't get duplicated cache flushes. In that sense, it's an slight performance improvement. Suggested-by: David Gibson Reviewed-by: Eric Auger Reviewed-by: Michael S. Tsirkin Acked-by: Alex Williamson Signed-off-by: Peter Xu Message-Id: <1491562755-23867-2-git-send-email-peterx@redhat.com> [ehabkost: included extra vhost_iommu_region_del() change from Peter Xu] Signed-off-by: Eduardo Habkost --- include/exec/memory.h | 19 ++++++++++++++++++- hw/vfio/common.c | 12 +++++++++--- hw/virtio/vhost.c | 13 ++++++++++--- memory.c | 9 +++++++++ 4 files changed, 46 insertions(+), 7 deletions(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index f20b191793..0840c89489 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -77,13 +77,30 @@ typedef enum { =20 #define IOMMU_NOTIFIER_ALL (IOMMU_NOTIFIER_MAP | IOMMU_NOTIFIER_UNMAP) =20 +struct IOMMUNotifier; +typedef void (*IOMMUNotify)(struct IOMMUNotifier *notifier, + IOMMUTLBEntry *data); + struct IOMMUNotifier { - void (*notify)(struct IOMMUNotifier *notifier, IOMMUTLBEntry *data); + IOMMUNotify notify; IOMMUNotifierFlag notifier_flags; + /* Notify for address space range start <=3D addr <=3D end */ + hwaddr start; + hwaddr end; QLIST_ENTRY(IOMMUNotifier) node; }; typedef struct IOMMUNotifier IOMMUNotifier; =20 +static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, + IOMMUNotifierFlag flags, + hwaddr start, hwaddr end) +{ + n->notify =3D fn; + n->notifier_flags =3D flags; + n->start =3D start; + n->end =3D end; +} + /* New-style MMIO accessors can indicate that the transaction failed. * A zero (MEMTX_OK) response means success; anything else is a failure * of some kind. The memory subsystem will bitwise-OR together results diff --git a/hw/vfio/common.c b/hw/vfio/common.c index f3ba9b9007..6b33b9f55d 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -478,8 +478,13 @@ static void vfio_listener_region_add(MemoryListener *l= istener, giommu->iommu_offset =3D section->offset_within_address_space - section->offset_within_region; giommu->container =3D container; - giommu->n.notify =3D vfio_iommu_map_notify; - giommu->n.notifier_flags =3D IOMMU_NOTIFIER_ALL; + llend =3D int128_add(int128_make64(section->offset_within_region), + section->size); + llend =3D int128_sub(llend, int128_one()); + iommu_notifier_init(&giommu->n, vfio_iommu_map_notify, + IOMMU_NOTIFIER_ALL, + section->offset_within_region, + int128_get64(llend)); QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); =20 memory_region_register_iommu_notifier(giommu->iommu, &giommu->n); @@ -550,7 +555,8 @@ static void vfio_listener_region_del(MemoryListener *li= stener, VFIOGuestIOMMU *giommu; =20 QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) { - if (giommu->iommu =3D=3D section->mr) { + if (giommu->iommu =3D=3D section->mr && + giommu->n.start =3D=3D section->offset_within_region) { memory_region_unregister_iommu_notifier(giommu->iommu, &giommu->n); QLIST_REMOVE(giommu, giommu_next); diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 613494dcc2..0001e60b77 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -736,14 +736,20 @@ static void vhost_iommu_region_add(MemoryListener *li= stener, struct vhost_dev *dev =3D container_of(listener, struct vhost_dev, iommu_listener); struct vhost_iommu *iommu; + Int128 end; =20 if (!memory_region_is_iommu(section->mr)) { return; } =20 iommu =3D g_malloc0(sizeof(*iommu)); - iommu->n.notify =3D vhost_iommu_unmap_notify; - iommu->n.notifier_flags =3D IOMMU_NOTIFIER_UNMAP; + end =3D int128_add(int128_make64(section->offset_within_region), + section->size); + end =3D int128_sub(end, int128_one()); + iommu_notifier_init(&iommu->n, vhost_iommu_unmap_notify, + IOMMU_NOTIFIER_UNMAP, + section->offset_within_region, + int128_get64(end)); iommu->mr =3D section->mr; iommu->iommu_offset =3D section->offset_within_address_space - section->offset_within_region; @@ -765,7 +771,8 @@ static void vhost_iommu_region_del(MemoryListener *list= ener, } =20 QLIST_FOREACH(iommu, &dev->iommu_list, iommu_next) { - if (iommu->mr =3D=3D section->mr) { + if (iommu->mr =3D=3D section->mr && + iommu->n.start =3D=3D section->offset_within_region) { memory_region_unregister_iommu_notifier(iommu->mr, &iommu->n); QLIST_REMOVE(iommu, iommu_next); diff --git a/memory.c b/memory.c index 4c95aaf39c..75ac595073 100644 --- a/memory.c +++ b/memory.c @@ -1606,6 +1606,7 @@ void memory_region_register_iommu_notifier(MemoryRegi= on *mr, =20 /* We need to register for at least one bitfield */ assert(n->notifier_flags !=3D IOMMU_NOTIFIER_NONE); + assert(n->start <=3D n->end); QLIST_INSERT_HEAD(&mr->iommu_notify, n, node); memory_region_update_iommu_notify_flags(mr); } @@ -1667,6 +1668,14 @@ void memory_region_notify_iommu(MemoryRegion *mr, } =20 QLIST_FOREACH(iommu_notifier, &mr->iommu_notify, node) { + /* + * Skip the notification if the notification does not overlap + * with registered range. + */ + if (iommu_notifier->start > entry.iova + entry.addr_mask + 1 || + iommu_notifier->end < entry.iova) { + continue; + } if (iommu_notifier->notifier_flags & request_flags) { iommu_notifier->notify(iommu_notifier, &entry); } --=20 2.11.0.259.g40922b1