From: Leon Romanovsky <leonro@nvidia.com>
Some exporters need a flow to synchronously revoke access to the DMA-buf
by importers. Once revoke is completed the importer is not permitted to
touch the memory otherwise they may get IOMMU faults, AERs, or worse.
DMA-buf today defines a revoke flow, for both pinned and dynamic
importers, which is broadly:
dma_resv_lock(dmabuf->resv, NULL);
// Prevent new mappings from being established
priv->revoked = true;
// Tell all importers to eventually unmap
dma_buf_invalidate_mappings(dmabuf);
// Wait for any inprogress fences on the old mapping
dma_resv_wait_timeout(dmabuf->resv,
DMA_RESV_USAGE_BOOKKEEP, false,
MAX_SCHEDULE_TIMEOUT);
dma_resv_unlock(dmabuf->resv, NULL);
// Wait for all importers to complete unmap
wait_for_completion(&priv->unmapped_comp);
This works well, and an importer that continues to access the DMA-buf
after unmapping it is very buggy.
However, the final wait for unmap is effectively unbounded. Several
importers do not support invalidate_mappings() at all and won't unmap
until userspace triggers it.
This unbounded wait is not suitable for exporters like VFIO and RDMA tha
need to issue revoke as part of their normal operations.
Add dma_buf_attach_revocable() to allow exporters to determine the
difference between importers that can complete the above in bounded time,
and those that can't. It can be called inside the exporter's attach op to
reject incompatible importers.
Document these details about how dma_buf_invalidate_mappings() works and
what the required sequence is to achieve a full revocation.
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
drivers/dma-buf/dma-buf.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++-
include/linux/dma-buf.h | 9 +++------
2 files changed, 50 insertions(+), 7 deletions(-)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index 1629312d364a..f0e05227bda8 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -1242,13 +1242,59 @@ void dma_buf_unmap_attachment_unlocked(struct dma_buf_attachment *attach,
}
EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_unlocked, "DMA_BUF");
+/**
+ * dma_buf_attach_revocable - check if a DMA-buf importer implements
+ * revoke semantics.
+ * @attach: the DMA-buf attachment to check
+ *
+ * Returns true if the DMA-buf importer can support the revoke sequence
+ * explained in dma_buf_invalidate_mappings() within bounded time. Meaning the
+ * importer implements invalidate_mappings() and ensures that unmap is called as
+ * a result.
+ */
+bool dma_buf_attach_revocable(struct dma_buf_attachment *attach)
+{
+ return attach->importer_ops &&
+ attach->importer_ops->invalidate_mappings;
+}
+EXPORT_SYMBOL_NS_GPL(dma_buf_attach_revocable, "DMA_BUF");
+
/**
* dma_buf_invalidate_mappings - notify attachments that DMA-buf is moving
*
* @dmabuf: [in] buffer which is moving
*
* Informs all attachments that they need to destroy and recreate all their
- * mappings.
+ * mappings. If the attachment is dynamic then the dynamic importer is expected
+ * to invalidate any caches it has of the mapping result and perform a new
+ * mapping request before allowing HW to do any further DMA.
+ *
+ * If the attachment is pinned then this informs the pinned importer that the
+ * underlying mapping is no longer available. Pinned importers may take this is
+ * as a permanent revocation and never establish new mappings so exporters
+ * should not trigger it lightly.
+ *
+ * Upon return importers may continue to access the DMA-buf memory. The caller
+ * must do two additional waits to ensure that the memory is no longer being
+ * accessed:
+ * 1) Until dma_resv_wait_timeout() retires fences the importer is allowed to
+ * fully access the memory.
+ * 2) Until the importer calls unmap it is allowed to speculatively
+ * read-and-discard the memory. It must not write to the memory.
+ *
+ * A caller wishing to use dma_buf_invalidate_mappings() to fully stop access to
+ * the DMA-buf must wait for both. Dynamic callers can often use just the first.
+ *
+ * All importers providing a invalidate_mappings() op must ensure that unmap is
+ * called within bounded time after the op.
+ *
+ * Pinned importers that do not support a invalidate_mappings() op will
+ * eventually perform unmap when they are done with the buffer, which may be an
+ * ubounded time from calling this function. dma_buf_attach_revocable() can be
+ * used to prevent such importers from attaching.
+ *
+ * Importers are free to request a new mapping in parallel as this function
+ * returns.
*/
void dma_buf_invalidate_mappings(struct dma_buf *dmabuf)
{
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index d5c3ce2b3aa4..84a7ec8f5359 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -468,12 +468,8 @@ struct dma_buf_attach_ops {
* called with this lock held as well. This makes sure that no mapping
* is created concurrently with an ongoing move operation.
*
- * Mappings stay valid and are not directly affected by this callback.
- * But the DMA-buf can now be in a different physical location, so all
- * mappings should be destroyed and re-created as soon as possible.
- *
- * New mappings can be created after this callback returns, and will
- * point to the new location of the DMA-buf.
+ * See the kdoc for dma_buf_invalidate_mappings() for details on the
+ * required behavior.
*/
void (*invalidate_mappings)(struct dma_buf_attachment *attach);
};
@@ -601,6 +597,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,
enum dma_data_direction);
void dma_buf_invalidate_mappings(struct dma_buf *dma_buf);
+bool dma_buf_attach_revocable(struct dma_buf_attachment *attach);
int dma_buf_begin_cpu_access(struct dma_buf *dma_buf,
enum dma_data_direction dir);
int dma_buf_end_cpu_access(struct dma_buf *dma_buf,
--
2.52.0On 1/24/26 20:14, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
>
> Some exporters need a flow to synchronously revoke access to the DMA-buf
> by importers. Once revoke is completed the importer is not permitted to
> touch the memory otherwise they may get IOMMU faults, AERs, or worse.
That approach is seriously not going to fly.
You can use the invalidate_mappings approach to trigger the importer to give back the mapping, but when the mapping is really given back is still completely on the importer side.
In other words you can't do the shot down revoke semantics you are trying to establish here.
Regards,
Christian.
>
> DMA-buf today defines a revoke flow, for both pinned and dynamic
> importers, which is broadly:
>
> dma_resv_lock(dmabuf->resv, NULL);
> // Prevent new mappings from being established
> priv->revoked = true;
>
> // Tell all importers to eventually unmap
> dma_buf_invalidate_mappings(dmabuf);
>
> // Wait for any inprogress fences on the old mapping
> dma_resv_wait_timeout(dmabuf->resv,
> DMA_RESV_USAGE_BOOKKEEP, false,
> MAX_SCHEDULE_TIMEOUT);
> dma_resv_unlock(dmabuf->resv, NULL);
>
> // Wait for all importers to complete unmap
> wait_for_completion(&priv->unmapped_comp);
>
> This works well, and an importer that continues to access the DMA-buf
> after unmapping it is very buggy.
>
> However, the final wait for unmap is effectively unbounded. Several
> importers do not support invalidate_mappings() at all and won't unmap
> until userspace triggers it.
>
> This unbounded wait is not suitable for exporters like VFIO and RDMA tha
> need to issue revoke as part of their normal operations.
>
> Add dma_buf_attach_revocable() to allow exporters to determine the
> difference between importers that can complete the above in bounded time,
> and those that can't. It can be called inside the exporter's attach op to
> reject incompatible importers.
>
> Document these details about how dma_buf_invalidate_mappings() works and
> what the required sequence is to achieve a full revocation.
>
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> ---
> drivers/dma-buf/dma-buf.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++-
> include/linux/dma-buf.h | 9 +++------
> 2 files changed, 50 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index 1629312d364a..f0e05227bda8 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
> @@ -1242,13 +1242,59 @@ void dma_buf_unmap_attachment_unlocked(struct dma_buf_attachment *attach,
> }
> EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_unlocked, "DMA_BUF");
>
> +/**
> + * dma_buf_attach_revocable - check if a DMA-buf importer implements
> + * revoke semantics.
> + * @attach: the DMA-buf attachment to check
> + *
> + * Returns true if the DMA-buf importer can support the revoke sequence
> + * explained in dma_buf_invalidate_mappings() within bounded time. Meaning the
> + * importer implements invalidate_mappings() and ensures that unmap is called as
> + * a result.
> + */
> +bool dma_buf_attach_revocable(struct dma_buf_attachment *attach)
> +{
> + return attach->importer_ops &&
> + attach->importer_ops->invalidate_mappings;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_attach_revocable, "DMA_BUF");
> +
> /**
> * dma_buf_invalidate_mappings - notify attachments that DMA-buf is moving
> *
> * @dmabuf: [in] buffer which is moving
> *
> * Informs all attachments that they need to destroy and recreate all their
> - * mappings.
> + * mappings. If the attachment is dynamic then the dynamic importer is expected
> + * to invalidate any caches it has of the mapping result and perform a new
> + * mapping request before allowing HW to do any further DMA.
> + *
> + * If the attachment is pinned then this informs the pinned importer that the
> + * underlying mapping is no longer available. Pinned importers may take this is
> + * as a permanent revocation and never establish new mappings so exporters
> + * should not trigger it lightly.
> + *
> + * Upon return importers may continue to access the DMA-buf memory. The caller
> + * must do two additional waits to ensure that the memory is no longer being
> + * accessed:
> + * 1) Until dma_resv_wait_timeout() retires fences the importer is allowed to
> + * fully access the memory.
> + * 2) Until the importer calls unmap it is allowed to speculatively
> + * read-and-discard the memory. It must not write to the memory.
> + *
> + * A caller wishing to use dma_buf_invalidate_mappings() to fully stop access to
> + * the DMA-buf must wait for both. Dynamic callers can often use just the first.
> + *
> + * All importers providing a invalidate_mappings() op must ensure that unmap is
> + * called within bounded time after the op.
> + *
> + * Pinned importers that do not support a invalidate_mappings() op will
> + * eventually perform unmap when they are done with the buffer, which may be an
> + * ubounded time from calling this function. dma_buf_attach_revocable() can be
> + * used to prevent such importers from attaching.
> + *
> + * Importers are free to request a new mapping in parallel as this function
> + * returns.
> */
> void dma_buf_invalidate_mappings(struct dma_buf *dmabuf)
> {
> diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> index d5c3ce2b3aa4..84a7ec8f5359 100644
> --- a/include/linux/dma-buf.h
> +++ b/include/linux/dma-buf.h
> @@ -468,12 +468,8 @@ struct dma_buf_attach_ops {
> * called with this lock held as well. This makes sure that no mapping
> * is created concurrently with an ongoing move operation.
> *
> - * Mappings stay valid and are not directly affected by this callback.
> - * But the DMA-buf can now be in a different physical location, so all
> - * mappings should be destroyed and re-created as soon as possible.
> - *
> - * New mappings can be created after this callback returns, and will
> - * point to the new location of the DMA-buf.
> + * See the kdoc for dma_buf_invalidate_mappings() for details on the
> + * required behavior.
> */
> void (*invalidate_mappings)(struct dma_buf_attachment *attach);
> };
> @@ -601,6 +597,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *,
> void dma_buf_unmap_attachment(struct dma_buf_attachment *, struct sg_table *,
> enum dma_data_direction);
> void dma_buf_invalidate_mappings(struct dma_buf *dma_buf);
> +bool dma_buf_attach_revocable(struct dma_buf_attachment *attach);
> int dma_buf_begin_cpu_access(struct dma_buf *dma_buf,
> enum dma_data_direction dir);
> int dma_buf_end_cpu_access(struct dma_buf *dma_buf,
>
On Fri, Jan 30, 2026 at 09:43:22AM +0100, Christian König wrote: > On 1/24/26 20:14, Leon Romanovsky wrote: > > From: Leon Romanovsky <leonro@nvidia.com> > > > > Some exporters need a flow to synchronously revoke access to the DMA-buf > > by importers. Once revoke is completed the importer is not permitted to > > touch the memory otherwise they may get IOMMU faults, AERs, or worse. > > That approach is seriously not going to fly. > > You can use the invalidate_mappings approach to trigger the importer > to give back the mapping, but when the mapping is really given back > is still completely on the importer side. Yes, and that is what this is all doing, there is the wait for the importer's unmap to happen in the sequence. > In other words you can't do the shot down revoke semantics you are > trying to establish here. All this is doing is saying if dma_buf_attach_revocable() == true then the importer will call unmap within bounded time after dma_buf_invalidate_mappings(). That's it. If the importing driver doesn't want to do that then it should make dma_buf_attach_revocable()=false. VFIO/etc only want to interwork with importers that can do this. Jason
Hi Leon,
On Sat, Jan 24, 2026 at 09:14:18PM +0200, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
>
> Some exporters need a flow to synchronously revoke access to the DMA-buf
> by importers. Once revoke is completed the importer is not permitted to
> touch the memory otherwise they may get IOMMU faults, AERs, or worse.
>
> DMA-buf today defines a revoke flow, for both pinned and dynamic
> importers, which is broadly:
>
> dma_resv_lock(dmabuf->resv, NULL);
> // Prevent new mappings from being established
> priv->revoked = true;
>
> // Tell all importers to eventually unmap
> dma_buf_invalidate_mappings(dmabuf);
>
> // Wait for any inprogress fences on the old mapping
> dma_resv_wait_timeout(dmabuf->resv,
> DMA_RESV_USAGE_BOOKKEEP, false,
> MAX_SCHEDULE_TIMEOUT);
> dma_resv_unlock(dmabuf->resv, NULL);
>
> // Wait for all importers to complete unmap
> wait_for_completion(&priv->unmapped_comp);
>
> This works well, and an importer that continues to access the DMA-buf
> after unmapping it is very buggy.
>
> However, the final wait for unmap is effectively unbounded. Several
> importers do not support invalidate_mappings() at all and won't unmap
> until userspace triggers it.
>
> This unbounded wait is not suitable for exporters like VFIO and RDMA tha
> need to issue revoke as part of their normal operations.
>
> Add dma_buf_attach_revocable() to allow exporters to determine the
> difference between importers that can complete the above in bounded time,
> and those that can't. It can be called inside the exporter's attach op to
> reject incompatible importers.
>
> Document these details about how dma_buf_invalidate_mappings() works and
> what the required sequence is to achieve a full revocation.
>
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> ---
> drivers/dma-buf/dma-buf.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++-
> include/linux/dma-buf.h | 9 +++------
> 2 files changed, 50 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index 1629312d364a..f0e05227bda8 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
> @@ -1242,13 +1242,59 @@ void dma_buf_unmap_attachment_unlocked(struct dma_buf_attachment *attach,
> }
> EXPORT_SYMBOL_NS_GPL(dma_buf_unmap_attachment_unlocked, "DMA_BUF");
>
> +/**
> + * dma_buf_attach_revocable - check if a DMA-buf importer implements
> + * revoke semantics.
> + * @attach: the DMA-buf attachment to check
> + *
> + * Returns true if the DMA-buf importer can support the revoke sequence
> + * explained in dma_buf_invalidate_mappings() within bounded time. Meaning the
> + * importer implements invalidate_mappings() and ensures that unmap is called as
> + * a result.
> + */
> +bool dma_buf_attach_revocable(struct dma_buf_attachment *attach)
> +{
> + return attach->importer_ops &&
> + attach->importer_ops->invalidate_mappings;
> +}
> +EXPORT_SYMBOL_NS_GPL(dma_buf_attach_revocable, "DMA_BUF");
> +
I noticed that Patch 5 removes the invalidate_mappings stub from
umem_dmabuf.c, effectively making the callback NULL for an RDMA
importer. Consequently, dma_buf_attach_revocable() (introduced here)
will return false for these importers.
Since the cover letter mentions that VFIO will use
dma_buf_attach_revocable() to prevent unbounded waits, this appears to
effectively block paths like the VFIO-export -> RDMA-import path..
Given that RDMA is a significant consumer of dma-bufs, are there plans
to implement proper revocation support in the IB/RDMA core (umem_dmabuf)?
It would be good to know if there's a plan for bringing such importers
into compliance with the new revocation semantics so they can interop
with VFIO OR are we completely ruling out users like RDMA / IB importing
any DMABUFs exported by VFIO?
Thanks,
Praan
On Mon, Jan 26, 2026 at 08:38:44PM +0000, Pranjal Shrivastava wrote: > I noticed that Patch 5 removes the invalidate_mappings stub from > umem_dmabuf.c, effectively making the callback NULL for an RDMA > importer. Consequently, dma_buf_attach_revocable() (introduced here) > will return false for these importers. Yes, that is the intention. > Since the cover letter mentions that VFIO will use > dma_buf_attach_revocable() to prevent unbounded waits, this appears to > effectively block paths like the VFIO-export -> RDMA-import path.. It remains usable with the ODP path and people are using that right now. > Given that RDMA is a significant consumer of dma-bufs, are there plans > to implement proper revocation support in the IB/RDMA core (umem_dmabuf)? This depends on each HW, they need a way to implement the revoke semantic. I can't guess what is possible, but I would hope that most HW could at least do a revoke on a real MR. Eg a MR rereg operation to a kernel owned empty PD is an effective "revoke", and MR rereg is at least defined by standards so HW should implement it. > It would be good to know if there's a plan for bringing such importers > into compliance with the new revocation semantics so they can interop > with VFIO OR are we completely ruling out users like RDMA / IB importing > any DMABUFs exported by VFIO? It will be driver dependent, there is no one shot update here. Jason
© 2016 - 2026 Red Hat, Inc.