s390_pci_iommu_disable() destroys the I/O TLB and unpins the host's
mappings, but it doesn't account for reclaiming the freed DMA slots.
This is problematic when an IOMMU is disabled and re-enabled while
reusing the container (e.g. during a guest reboot), which may cause the
RPCIT path to fail with DMA slots exhaustion errors, manifesting as
unexplained ENOMEMs in the guest.

Fix it by reclaiming all the entries in the I/O TLB's hash table before
removing all its entries. The actual DMA slots consumed by the IOMMU are
already relinquished to vfio when the memory subregion is deleted in the
same function.

Cc: qemu-stable@nongnu.org
Fixes: 37fa32de7073 ("s390x/pci: Honor DMA limits set by vfio")
Signed-off-by: Omar Elghoul <oelghoul@linux.ibm.com>
---
 hw/s390x/s390-pci-bus.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c
index 2724b27ec2..cdd75cdfd2 100644
--- a/hw/s390x/s390-pci-bus.c
+++ b/hw/s390x/s390-pci-bus.c
@@ -826,6 +826,14 @@ void s390_pci_iommu_disable(S390PCIBusDevice *pbdev)
 {
     S390PCIIOMMU *iommu = pbdev->iommu;
     pbdev->iommu_enabled = false;
+    if (pbdev->dma_limit) {
+        /*
+         * Account for all the DMA mappings still in the IOTLB; they will be
+         * returned to vfio when we delete the memory subregion via
+         * memory_region_del_subregion() below.
+         */
+        pbdev->dma_limit->avail += g_hash_table_size(pbdev->iotlb);
+    }
     g_hash_table_remove_all(pbdev->iotlb);
     if (pbdev->dm_mr) {
         memory_region_del_subregion(&iommu->mr, pbdev->dm_mr);
-- 
2.56.0

On 10/21/26 1:50 PM, Omar Elghoul wrote:
> s390_pci_iommu_disable() destroys the I/O TLB and unpins the host's
> mappings, but it doesn't account for reclaiming the freed DMA slots.
> This is problematic when an IOMMU is disabled and re-enabled while
> reusing the container (e.g. during a guest reboot), which may cause the
> RPCIT path to fail with DMA slots exhaustion errors, manifesting as
> unexplained ENOMEMs in the guest.
> 
> Fix it by reclaiming all the entries in the I/O TLB's hash table before
> removing all its entries. The actual DMA slots consumed by the IOMMU are
> already relinquished to vfio when the memory subregion is deleted in the
> same function.
> 
> Cc: qemu-stable@nongnu.org
> Fixes: 37fa32de7073 ("s390x/pci: Honor DMA limits set by vfio")
> Signed-off-by: Omar Elghoul <oelghoul@linux.ibm.com>
> ---
>   hw/s390x/s390-pci-bus.c | 8 ++++++++
>   1 file changed, 8 insertions(+)

Applied this to my queue for 11.2; thank you!

On 10/21/26 1:50 PM, Omar Elghoul wrote:
> s390_pci_iommu_disable() destroys the I/O TLB and unpins the host's
> mappings, but it doesn't account for reclaiming the freed DMA slots.
> This is problematic when an IOMMU is disabled and re-enabled while
> reusing the container (e.g. during a guest reboot), which may cause the
> RPCIT path to fail with DMA slots exhaustion errors, manifesting as
> unexplained ENOMEMs in the guest.
> 
> Fix it by reclaiming all the entries in the I/O TLB's hash table before
> removing all its entries. The actual DMA slots consumed by the IOMMU are
> already relinquished to vfio when the memory subregion is deleted in the
> same function.
> 
> Cc: qemu-stable@nongnu.org
> Fixes: 37fa32de7073 ("s390x/pci: Honor DMA limits set by vfio")
> Signed-off-by: Omar Elghoul <oelghoul@linux.ibm.com>

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Tested-by: Matthew Rosato <mjrosato@linux.ibm.com>

Thanks Omar, I like this alot better for a fix!  It is minimal and
accounts for exactly what was not getting reclaimed.

This approach does lose the re-validation with the vfio-reported value
as compared to your prior approach, which might be handy for detecting
any other leaks in the future; but that is something we could consider
adding as a non-fix add-on later.

Thanks,
Matt

> ---
>  hw/s390x/s390-pci-bus.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c
> index 2724b27ec2..cdd75cdfd2 100644
> --- a/hw/s390x/s390-pci-bus.c
> +++ b/hw/s390x/s390-pci-bus.c
> @@ -826,6 +826,14 @@ void s390_pci_iommu_disable(S390PCIBusDevice *pbdev)
>  {
>      S390PCIIOMMU *iommu = pbdev->iommu;
>      pbdev->iommu_enabled = false;
> +    if (pbdev->dma_limit) {
> +        /*
> +         * Account for all the DMA mappings still in the IOTLB; they will be
> +         * returned to vfio when we delete the memory subregion via
> +         * memory_region_del_subregion() below.
> +         */
> +        pbdev->dma_limit->avail += g_hash_table_size(pbdev->iotlb);
> +    }
>      g_hash_table_remove_all(pbdev->iotlb);
>      if (pbdev->dm_mr) {
>          memory_region_del_subregion(&iommu->mr, pbdev->dm_mr);

On 10/21/26 1:50 PM, Omar Elghoul wrote:
> s390_pci_iommu_disable() destroys the I/O TLB and unpins the host's
> mappings, but it doesn't account for reclaiming the freed DMA slots.
> This is problematic when an IOMMU is disabled and re-enabled while
> reusing the container (e.g. during a guest reboot), which may cause the
> RPCIT path to fail with DMA slots exhaustion errors, manifesting as
> unexplained ENOMEMs in the guest.
> 
> Fix it by reclaiming all the entries in the I/O TLB's hash table before
> removing all its entries. The actual DMA slots consumed by the IOMMU are
> already relinquished to vfio when the memory subregion is deleted in the
> same function.
> 
> Cc: qemu-stable@nongnu.org
> Fixes: 37fa32de7073 ("s390x/pci: Honor DMA limits set by vfio")
> Signed-off-by: Omar Elghoul <oelghoul@linux.ibm.com>
For context, this patch addresses the same issue as the patch linked
below [1], but after some internal discussion, it was decided that this
approach is safer than overwriting the container's max DMA limit and
current DMA usage every time an IOMMU gets disabled.

The reasoning is that multiple devices can, in theory at least, share
the same vfio container and IOMMU. This isn't true for zpci devices in
practice today, but the code is already built to handle that case.

[1] 
https://lore.kernel.org/all/20261019153729.40823-1-oelghoul@linux.ibm.com/