Even if Device-TLB and PCI ATS is enabled, the guest can reject to use
it. For example, this situation appears when Windows Server 2022 is
running with intel-iommu with device-iotlb=on and virtio-net-pci with
vhost=on. The guest implies that no address translation info cached in
device IOTLB and doesn't send device IOTLB invalidation commands. So,
it leads to irrelevant address translations in vhost-net in the host
kernel. Therefore network frames from the guest in host tap interface
contains wrong payload data.
This patch enables IOTLB unmap events (IOMMU_NOTIFIER_UNMAP) along with
Device-TLB unmap events (IOMMU_NOTIFIER_DEVIOTLB_UNMAP) handling for
proper vhost IOTLB unmapping when the guest isn't aware of Device-TLB.
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2001312
Signed-off-by: Viktor Prutyanov <viktor@daynix.com>
---
Tested on Windows Server 2022 and Fedora guests with
-device virtio-net-pci,bus=pci.3,netdev=nd0,iommu_platform=on,ats=on
-netdev tap,id=nd0,ifname=tap1,script=no,downscript=no,vhost=on
-device intel-iommu,intremap=on,eim=on,device-iotlb=on/off
hw/virtio/vhost.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index a266396576..968ca18fce 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -796,7 +796,7 @@ static void vhost_iommu_region_add(MemoryListener *listener,
iommu_idx = memory_region_iommu_attrs_to_index(iommu_mr,
MEMTXATTRS_UNSPECIFIED);
iommu_notifier_init(&iommu->n, vhost_iommu_unmap_notify,
- IOMMU_NOTIFIER_DEVIOTLB_UNMAP,
+ IOMMU_NOTIFIER_DEVIOTLB_UNMAP | IOMMU_NOTIFIER_UNMAP,
section->offset_within_region,
int128_get64(end),
iommu_idx);
--
2.35.1On Thu, Mar 30, 2023 at 7:49 PM Viktor Prutyanov <viktor@daynix.com> wrote: > > Even if Device-TLB and PCI ATS is enabled, the guest can reject to use > it. For example, this situation appears when Windows Server 2022 is > running with intel-iommu with device-iotlb=on and virtio-net-pci with > vhost=on. The guest implies that no address translation info cached in > device IOTLB and doesn't send device IOTLB invalidation commands. So, > it leads to irrelevant address translations in vhost-net in the host > kernel. Therefore network frames from the guest in host tap interface > contains wrong payload data. > > This patch enables IOTLB unmap events (IOMMU_NOTIFIER_UNMAP) along with > Device-TLB unmap events (IOMMU_NOTIFIER_DEVIOTLB_UNMAP) handling for > proper vhost IOTLB unmapping when the guest isn't aware of Device-TLB. > > Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2001312 > Signed-off-by: Viktor Prutyanov <viktor@daynix.com> > --- > > Tested on Windows Server 2022 and Fedora guests with > -device virtio-net-pci,bus=pci.3,netdev=nd0,iommu_platform=on,ats=on > -netdev tap,id=nd0,ifname=tap1,script=no,downscript=no,vhost=on > -device intel-iommu,intremap=on,eim=on,device-iotlb=on/off > > hw/virtio/vhost.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c > index a266396576..968ca18fce 100644 > --- a/hw/virtio/vhost.c > +++ b/hw/virtio/vhost.c > @@ -796,7 +796,7 @@ static void vhost_iommu_region_add(MemoryListener *listener, > iommu_idx = memory_region_iommu_attrs_to_index(iommu_mr, > MEMTXATTRS_UNSPECIFIED); > iommu_notifier_init(&iommu->n, vhost_iommu_unmap_notify, > - IOMMU_NOTIFIER_DEVIOTLB_UNMAP, > + IOMMU_NOTIFIER_DEVIOTLB_UNMAP | IOMMU_NOTIFIER_UNMAP, > section->offset_within_region, > int128_get64(end), > iommu_idx); > -- > 2.35.1 > ping
On Thu, Apr 13, 2023 at 2:24 PM Viktor Prutyanov <viktor@daynix.com> wrote: > > On Thu, Mar 30, 2023 at 7:49 PM Viktor Prutyanov <viktor@daynix.com> wrote: > > > > Even if Device-TLB and PCI ATS is enabled, the guest can reject to use > > it. For example, this situation appears when Windows Server 2022 is > > running with intel-iommu with device-iotlb=on and virtio-net-pci with > > vhost=on. The guest implies that no address translation info cached in > > device IOTLB and doesn't send device IOTLB invalidation commands. So, > > it leads to irrelevant address translations in vhost-net in the host > > kernel. Therefore network frames from the guest in host tap interface > > contains wrong payload data. > > > > This patch enables IOTLB unmap events (IOMMU_NOTIFIER_UNMAP) along with > > Device-TLB unmap events (IOMMU_NOTIFIER_DEVIOTLB_UNMAP) handling for > > proper vhost IOTLB unmapping when the guest isn't aware of Device-TLB. > > > > Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2001312 > > Signed-off-by: Viktor Prutyanov <viktor@daynix.com> > > --- > > > > Tested on Windows Server 2022 and Fedora guests with > > -device virtio-net-pci,bus=pci.3,netdev=nd0,iommu_platform=on,ats=on > > -netdev tap,id=nd0,ifname=tap1,script=no,downscript=no,vhost=on > > -device intel-iommu,intremap=on,eim=on,device-iotlb=on/off > > > > hw/virtio/vhost.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c > > index a266396576..968ca18fce 100644 > > --- a/hw/virtio/vhost.c > > +++ b/hw/virtio/vhost.c > > @@ -796,7 +796,7 @@ static void vhost_iommu_region_add(MemoryListener *listener, > > iommu_idx = memory_region_iommu_attrs_to_index(iommu_mr, > > MEMTXATTRS_UNSPECIFIED); > > iommu_notifier_init(&iommu->n, vhost_iommu_unmap_notify, > > - IOMMU_NOTIFIER_DEVIOTLB_UNMAP, > > + IOMMU_NOTIFIER_DEVIOTLB_UNMAP | IOMMU_NOTIFIER_UNMAP, > > section->offset_within_region, > > int128_get64(end), > > iommu_idx); This looks kind of tricky. E.g when ATS is enabled, there could be unnecessary flues. Or at least it requires some comment to explain. I think the correct fix is: 1) introduce a transport specific method for querying if it supports device IOTLB 2) in the pci implementation hook it to some like pice_ats_enabled() 3) so we can decide if device IOTLB is enabled, go for DEVIOTLB_UNMAP otherwise a simple UNMAP. Thanks > > -- > > 2.35.1 > > > > ping >
© 2016 - 2024 Red Hat, Inc.