As restricted dma pools are always decrypted, in swiotlb.c it uses
phys_to_dma_unencrypted() for address conversion.

However, in DMA-direct, calls to phys_to_dma_direct() with
force_dma_unencrypted() returning false, will fallback to
phys_to_dma() which is inconsistent for memory allocated from
restricted dma pools.

Signed-off-by: Mostafa Saleh <smostafa@google.com>
---
 kernel/dma/direct.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 27d804f0473f..1a402bb956d9 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -26,7 +26,7 @@ u64 zone_dma_limit __ro_after_init = DMA_BIT_MASK(24);
 static inline dma_addr_t phys_to_dma_direct(struct device *dev,
 		phys_addr_t phys)
 {
-	if (force_dma_unencrypted(dev))
+	if (force_dma_unencrypted(dev) || is_swiotlb_for_alloc(dev))
 		return phys_to_dma_unencrypted(dev, phys);
 	return phys_to_dma(dev, phys);
 }
-- 
2.53.0.473.g4a7958ca14-goog

On Thu, Mar 05, 2026 at 05:03:35PM +0000, Mostafa Saleh wrote:
> As restricted dma pools are always decrypted, in swiotlb.c it uses
> phys_to_dma_unencrypted() for address conversion.
> 
> However, in DMA-direct, calls to phys_to_dma_direct() with
> force_dma_unencrypted() returning false, will fallback to
> phys_to_dma() which is inconsistent for memory allocated from
> restricted dma pools.
> 
> Signed-off-by: Mostafa Saleh <smostafa@google.com>
> ---
>  kernel/dma/direct.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
> index 27d804f0473f..1a402bb956d9 100644
> --- a/kernel/dma/direct.c
> +++ b/kernel/dma/direct.c
> @@ -26,7 +26,7 @@ u64 zone_dma_limit __ro_after_init = DMA_BIT_MASK(24);
>  static inline dma_addr_t phys_to_dma_direct(struct device *dev,
>  		phys_addr_t phys)
>  {
> -	if (force_dma_unencrypted(dev))
> +	if (force_dma_unencrypted(dev) || is_swiotlb_for_alloc(dev))
>  		return phys_to_dma_unencrypted(dev, phys);
>  	return phys_to_dma(dev, phys);
>  }

I couldn't fully get my head around the DMA API but I think all the
pools and bounce buffers are decrypted and protected guests (or realms
for Arm CCA) should always return true for force_dma_unencrypted(). If
that's the case, the above change wouldn't be necessary. I can see that
arm64 only does this for CCA and not pKVM guests.

Device assignment is another story that requires reworking those DMA
pools to support encrypted buffers.

-- 
Catalin

On Tue, Mar 10, 2026 at 01:08:00PM +0000, Catalin Marinas wrote:
> On Thu, Mar 05, 2026 at 05:03:35PM +0000, Mostafa Saleh wrote:
> > As restricted dma pools are always decrypted, in swiotlb.c it uses
> > phys_to_dma_unencrypted() for address conversion.
> > 
> > However, in DMA-direct, calls to phys_to_dma_direct() with
> > force_dma_unencrypted() returning false, will fallback to
> > phys_to_dma() which is inconsistent for memory allocated from
> > restricted dma pools.
> > 
> > Signed-off-by: Mostafa Saleh <smostafa@google.com>
> > ---
> >  kernel/dma/direct.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
> > index 27d804f0473f..1a402bb956d9 100644
> > --- a/kernel/dma/direct.c
> > +++ b/kernel/dma/direct.c
> > @@ -26,7 +26,7 @@ u64 zone_dma_limit __ro_after_init = DMA_BIT_MASK(24);
> >  static inline dma_addr_t phys_to_dma_direct(struct device *dev,
> >  		phys_addr_t phys)
> >  {
> > -	if (force_dma_unencrypted(dev))
> > +	if (force_dma_unencrypted(dev) || is_swiotlb_for_alloc(dev))
> >  		return phys_to_dma_unencrypted(dev, phys);
> >  	return phys_to_dma(dev, phys);
> >  }
> 
> I couldn't fully get my head around the DMA API but I think all the
> pools and bounce buffers are decrypted and protected guests (or realms
> for Arm CCA) should always return true for force_dma_unencrypted(). If
> that's the case, the above change wouldn't be necessary. I can see that
> arm64 only does this for CCA and not pKVM guests.
> 

Yes, that’s the problem, pKVM relies on SWIOTLB to use decrypted
buffers and not force_dma_unencrypted() in DMA-direct.
So, at the moment pKVM guests actually call:
- phys_to_dma_unencrypted(): From swiotlb code
- phys_to_dma(): From Direct-DMA code

Which is in-consistent, but only works as the pKVM memory encryption/
decryption is in-place, so there is no address conversion.

I was looking into setting force_dma_unencrypted() to true for pKVM,
which then resulted in the bug of double-decryption I am trying to solve
with patch-1.

I think the main problem is that SWIOTLB(restricted DMA) decrypts stuff
unconditionally, so we have to treat is_swiotlb_for_alloc() the same way as
force_dma_unencrypted().
That is what these 2 patches do, otherwise we teach SWIOTLB code about
force_dma_unencrypted().

Thanks,
Mostafa


> Device assignment is another story that requires reworking those DMA
> pools to support encrypted buffers.


> 
> -- 
> Catalin

On 10/03/2026 13:08, Catalin Marinas wrote:
> On Thu, Mar 05, 2026 at 05:03:35PM +0000, Mostafa Saleh wrote:
>> As restricted dma pools are always decrypted, in swiotlb.c it uses
>> phys_to_dma_unencrypted() for address conversion.
>>
>> However, in DMA-direct, calls to phys_to_dma_direct() with
>> force_dma_unencrypted() returning false, will fallback to
>> phys_to_dma() which is inconsistent for memory allocated from
>> restricted dma pools.
>>
>> Signed-off-by: Mostafa Saleh <smostafa@google.com>
>> ---
>>   kernel/dma/direct.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
>> index 27d804f0473f..1a402bb956d9 100644
>> --- a/kernel/dma/direct.c
>> +++ b/kernel/dma/direct.c
>> @@ -26,7 +26,7 @@ u64 zone_dma_limit __ro_after_init = DMA_BIT_MASK(24);
>>   static inline dma_addr_t phys_to_dma_direct(struct device *dev,
>>   		phys_addr_t phys)
>>   {
>> -	if (force_dma_unencrypted(dev))
>> +	if (force_dma_unencrypted(dev) || is_swiotlb_for_alloc(dev))
>>   		return phys_to_dma_unencrypted(dev, phys);
>>   	return phys_to_dma(dev, phys);
>>   }
> 
> I couldn't fully get my head around the DMA API but I think all the
> pools and bounce buffers are decrypted and protected guests (or realms
> for Arm CCA) should always return true for force_dma_unencrypted(). If
> that's the case, the above change wouldn't be necessary. I can see that
> arm64 only does this for CCA and not pKVM guests.

That is correct. Why would the force_dma_unencrypted() return false for
a device ?  As far as I can see, all CC guests are treating all devices
as "untrusted" for now (and there is a series available that is adding
support for "trusted devices [0]).


> 
> Device assignment is another story that requires reworking those DMA
> pools to support encrypted buffers.
> 

[0] 
https://lkml.kernel.org/r/20260303000207.1836586-1-dan.j.williams@intel.com

Suzuki