[PATCH 1/6] Revert "virtio-iommu: Clear IOMMUDevice when VFIO device is unplugged"

Eric Auger posted 6 patches 2 months ago
Maintainers: Alex Williamson <alex.williamson@redhat.com>, "Cédric Le Goater" <clg@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>, Eric Auger <eric.auger@redhat.com>
[PATCH 1/6] Revert "virtio-iommu: Clear IOMMUDevice when VFIO device is unplugged"
Posted by Eric Auger 2 months ago
This reverts commit 1b889d6e39c32d709f1114699a014b381bcf1cb1.
There are different problems with that tentative fix:
- Some resources are left dangling (resv_regions,
  host_resv_ranges) and memory subregions are left attached to
  the root MR although freed as embedded in the sdev IOMMUDevice.
  Finally the sdev->as is not destroyed and associated listeners
  are left.
- Even when fixing the above we observe a memory corruption
  associated with the deallocation of the IOMMUDevice. This can
  be observed when a VFIO device is hotplugged, hot-unplugged
  and a system reset is issued. At this stage we have not been
  able to identify the root cause (IOMMU MR or as structs beeing
  overwritten and used later on?).
- Another issue is HostIOMMUDevice are indexed by non aliased
  BDF whereas the IOMMUDevice is indexed by aliased BDF - yes the
  current naming is really misleading -. Given the state of the
  code I don't think the virtio-iommu device works in non
  singleton group case though.

So let's revert the patch for now. This means the IOMMU MR/as survive
the hotunplug. This is what is done in the intel_iommu for instance.
It does not sound very logical to keep those but currently there is
no symetric function to pci_device_iommu_address_space().

probe_done issue will be handled in a subsequent patch. Also
resv_regions and host_resv_regions will be deallocated separately.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
 hw/virtio/virtio-iommu.c | 21 ---------------------
 1 file changed, 21 deletions(-)

diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index 33ae61c4a6..4e34dacd6e 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -467,26 +467,6 @@ static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
     return &sdev->as;
 }
 
-static void virtio_iommu_device_clear(VirtIOIOMMU *s, PCIBus *bus, int devfn)
-{
-    IOMMUPciBus *sbus = g_hash_table_lookup(s->as_by_busptr, bus);
-    IOMMUDevice *sdev;
-
-    if (!sbus) {
-        return;
-    }
-
-    sdev = sbus->pbdev[devfn];
-    if (!sdev) {
-        return;
-    }
-
-    g_list_free_full(sdev->resv_regions, g_free);
-    sdev->resv_regions = NULL;
-    g_free(sdev);
-    sbus->pbdev[devfn] = NULL;
-}
-
 static gboolean hiod_equal(gconstpointer v1, gconstpointer v2)
 {
     const struct hiod_key *key1 = v1;
@@ -728,7 +708,6 @@ virtio_iommu_unset_iommu_device(PCIBus *bus, void *opaque, int devfn)
     }
 
     g_hash_table_remove(viommu->host_iommu_devices, &key);
-    virtio_iommu_device_clear(viommu, bus, devfn);
 }
 
 static const PCIIOMMUOps virtio_iommu_ops = {
-- 
2.41.0
Re: [PATCH 1/6] Revert "virtio-iommu: Clear IOMMUDevice when VFIO device is unplugged"
Posted by Cédric Le Goater 2 months ago
On 7/16/24 11:45, Eric Auger wrote:
> This reverts commit 1b889d6e39c32d709f1114699a014b381bcf1cb1.
> There are different problems with that tentative fix:
> - Some resources are left dangling (resv_regions,
>    host_resv_ranges) and memory subregions are left attached to
>    the root MR although freed as embedded in the sdev IOMMUDevice.
>    Finally the sdev->as is not destroyed and associated listeners
>    are left.
> - Even when fixing the above we observe a memory corruption
>    associated with the deallocation of the IOMMUDevice. This can
>    be observed when a VFIO device is hotplugged, hot-unplugged
>    and a system reset is issued. At this stage we have not been
>    able to identify the root cause (IOMMU MR or as structs beeing
>    overwritten and used later on?).
> - Another issue is HostIOMMUDevice are indexed by non aliased
>    BDF whereas the IOMMUDevice is indexed by aliased BDF - yes the
>    current naming is really misleading -. Given the state of the
>    code I don't think the virtio-iommu device works in non
>    singleton group case though.
> 
> So let's revert the patch for now. This means the IOMMU MR/as survive
> the hotunplug. This is what is done in the intel_iommu for instance.
> It does not sound very logical to keep those but currently there is
> no symetric function to pci_device_iommu_address_space().

Fully agree.
  
> probe_done issue will be handled in a subsequent patch. Also
> resv_regions and host_resv_regions will be deallocated separately.
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>


Reviewed-by: Cédric Le Goater <clg@redhat.com>

Thanks,

C.


> ---
>   hw/virtio/virtio-iommu.c | 21 ---------------------
>   1 file changed, 21 deletions(-)
> 
> diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
> index 33ae61c4a6..4e34dacd6e 100644
> --- a/hw/virtio/virtio-iommu.c
> +++ b/hw/virtio/virtio-iommu.c
> @@ -467,26 +467,6 @@ static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
>       return &sdev->as;
>   }
>   
> -static void virtio_iommu_device_clear(VirtIOIOMMU *s, PCIBus *bus, int devfn)
> -{
> -    IOMMUPciBus *sbus = g_hash_table_lookup(s->as_by_busptr, bus);
> -    IOMMUDevice *sdev;
> -
> -    if (!sbus) {
> -        return;
> -    }
> -
> -    sdev = sbus->pbdev[devfn];
> -    if (!sdev) {
> -        return;
> -    }
> -
> -    g_list_free_full(sdev->resv_regions, g_free);
> -    sdev->resv_regions = NULL;
> -    g_free(sdev);
> -    sbus->pbdev[devfn] = NULL;
> -}
> -
>   static gboolean hiod_equal(gconstpointer v1, gconstpointer v2)
>   {
>       const struct hiod_key *key1 = v1;
> @@ -728,7 +708,6 @@ virtio_iommu_unset_iommu_device(PCIBus *bus, void *opaque, int devfn)
>       }
>   
>       g_hash_table_remove(viommu->host_iommu_devices, &key);
> -    virtio_iommu_device_clear(viommu, bus, devfn);
>   }
>   
>   static const PCIIOMMUOps virtio_iommu_ops = {