Introduction
============
This is the third version of the fixes for direct-dma dealing with
memory encryption and restricted-dma.

Changes in v3:
- Instead of extending the logic by using is_swiotlb_for_alloc(),
  follow Jason’s suggestion and propagate the state of the memory
  allocated.
- Remove checks out of dma_set_*() based on Jason suggestion
- Remove documentation for now until we are close to the final
  proposal and add it later if needed.

Background
==========
At the moment the following hypervisor guests will need to deal with
memory encryption:
- pKVM (ARM): Documentation/virt/kvm/arm/hypercalls.rst
- ARM CCA: Documentation/arch/arm64/arm-cca.rst
- Intel TDX: Documentation/arch/x86/tdx.rst
- AMD SEV: Documentation/arch/x86/amd-memory-encryption.rst
- PPC SVM: Documentation/arch/powerpc/ultravisor.rst
- Hyper-V: Documentation/virt/hyperv/coco.rst

AFAICT, all (confidential) guests running under those have the memory
encrypted by default and guests will then explicitly share the memory
back if needed.

The main use cases for decrypting(sharing) memory are:
- Sharing memory back to the host through SWIOTLB (for virtio...)
- Hypervisor specific communication (ex: snp_msg, GHCB, VMBUS...)
- Shared/emulated resources: VGARAM (x86-SEV), GIC ITS tables (arm64)

While encrypting memory is typically used for reverting the
set_memory_decrypted() either in error handling or in freeing shared
resources back to the kernel.

Design
======
This series focuses mainly on dma-direct interaction with memory
encryption which is the complicated case.
At the moment memory encryption and dma-direct interacts in 2 ways:
1) force_dma_direct(): if true, memory will be decrypted by default
   on allocation.
2) Restricted DMA: where memory is pre-decrypted and managed by
   SWIOTLB.

With a third possible usage on the way [1] where the DMA-API allows
an attr for decrypted memory.

Instead of open coding many checks with is_swiotlb_for_alloc() and
force_dma_unencrypted().
Make __dma_direct_alloc_pages() return the state of allocated memory
encapsulated on the new internal type dma_page.
Then based on the memory state, dma-direct can identify what to do
based on the cases:
- Memory needs to be decrypted but is not: dma-direct will decrypt
  the memory and use the proper phys address conversions and page
  table prot.
- Memory is already decrypted: dma-direct will not decrypt the memory
  but it will use the proper phys address conversions and page table
  prot.

The free part is more tricky as we already lose the information about
allocation, so we have to check with each allocator separately, so
swiotlb_is_decrypted() is added for SWIOTLB which is only allocator
that can return decrypted memory.

Testing
=======
I was able to test this only under pKVM (arm64) as I have no
access to other systems.

Future work
===========
Two other things I am also looking at which are related to restricted
DMA pools, so they should be a different series.
1) Private pools: Currently all restricted DMA pools are decrypted
   (shared) by default. Having private pools would be useful for
   device assignment when bouncing is needed (as for non-coherent
   devices)
2) Optimizations for memory sharing. In some cases, allocations from
   restricted dma-pools are page aligned. For CoCo cases, that means
   that it will be cheaper to share memory in-place instead of
   bouncing.

Both of these add new semantics which need to be done carefully to
avoid regressions, and might be a good candidate for a topic in the
next LPC.

Patches
=======
- 1 Extend swiotlb
- 2-4 Refactoring
- 5 Fixes

v1: https://lore.kernel.org/all/20260305170335.963568-1-smostafa@google.com/
v2: https://lore.kernel.org/all/20260330145043.1586623-1-smostafa@google.com/

[1] https://lore.kernel.org/all/20260305123641.164164-1-jiri@resnulli.us/


Mostafa Saleh (5):
  swiotlb: Return state of memory from swiotlb_alloc()
  dma-mapping: Move encryption in __dma_direct_free_pages()
  dma-mapping: Decrypt memory on remap
  dma-mapping: Encapsulate memory state during allocation
  dma-mapping: Fix memory decryption issues

 include/linux/swiotlb.h |  25 +++++++-
 kernel/dma/direct.c     | 134 +++++++++++++++++++++++++++-------------
 kernel/dma/swiotlb.c    |  23 ++++++-
 3 files changed, 135 insertions(+), 47 deletions(-)

-- 
2.53.0.1213.gd9a14994de-goog

On Wed, Apr 08, 2026 at 07:47:37PM +0000, Mostafa Saleh wrote:
> Introduction
> ============
> This is the third version of the fixes for direct-dma dealing with
> memory encryption and restricted-dma.
> 
> Changes in v3:
> - Instead of extending the logic by using is_swiotlb_for_alloc(),
>   follow Jason’s suggestion and propagate the state of the memory
>   allocated.
> - Remove checks out of dma_set_*() based on Jason suggestion
> - Remove documentation for now until we are close to the final
>   proposal and add it later if needed.

There are a number of Sashiko remarks that look plausible that should
be investigated:
https://sashiko.dev/#/patchset/20260408194750.2280873-1-smostafa%40google.com

> Design
> ======
> This series focuses mainly on dma-direct interaction with memory
> encryption which is the complicated case.
> At the moment memory encryption and dma-direct interacts in 2 ways:
> 1) force_dma_direct(): if true, memory will be decrypted by default
>    on allocation.
> 2) Restricted DMA: where memory is pre-decrypted and managed by
>    SWIOTLB.
> 
> With a third possible usage on the way [1] where the DMA-API allows
> an attr for decrypted memory.

This [1] was merged now

Jason

On Fri, Apr 10, 2026 at 02:43:38PM -0300, Jason Gunthorpe wrote:
> On Wed, Apr 08, 2026 at 07:47:37PM +0000, Mostafa Saleh wrote:
> > Introduction
> > ============
> > This is the third version of the fixes for direct-dma dealing with
> > memory encryption and restricted-dma.
> > 
> > Changes in v3:
> > - Instead of extending the logic by using is_swiotlb_for_alloc(),
> >   follow Jason’s suggestion and propagate the state of the memory
> >   allocated.
> > - Remove checks out of dma_set_*() based on Jason suggestion
> > - Remove documentation for now until we are close to the final
> >   proposal and add it later if needed.
> 
> There are a number of Sashiko remarks that look plausible that should
> be investigated:
> https://sashiko.dev/#/patchset/20260408194750.2280873-1-smostafa%40google.com

I think the remap and NULL points are valid, I will address them.

The case of dma_coherent_ok() is more tricky, it is not a regression,
but I think it’s still a theoretical problem for some CCA solutions
where encrypted/decrypted memory have different DMA aliases. It’s not
easy to fix it without having some helper to check the memory state as
force_dma_unencrypted and swiotlb_is_decrypted() which kind of defeat
the purpose of returning the memory state from swiotlb_alloc() ://

> 
> > Design
> > ======
> > This series focuses mainly on dma-direct interaction with memory
> > encryption which is the complicated case.
> > At the moment memory encryption and dma-direct interacts in 2 ways:
> > 1) force_dma_direct(): if true, memory will be decrypted by default
> >    on allocation.
> > 2) Restricted DMA: where memory is pre-decrypted and managed by
> >    SWIOTLB.
> > 
> > With a third possible usage on the way [1] where the DMA-API allows
> > an attr for decrypted memory.
> 
> This [1] was merged now

I see, I will rebase on top of it and send v4.

Thanks,
Mostafa

> 
> Jason