[PATCH] swiotlb: iommu/dma: Clarify swiotlb options apply only to dma-direct

T.J. Mercier posted 1 patch 1 year, 7 months ago
Documentation/admin-guide/kernel-parameters.txt | 1 +
Documentation/arch/x86/x86_64/boot-options.rst  | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
[PATCH] swiotlb: iommu/dma: Clarify swiotlb options apply only to dma-direct
Posted by T.J. Mercier 1 year, 7 months ago
IOMMU implementations now sometimes bounce memory through SWIOTLB to
achieve cacheline alignment [1], or prevent DMA attacks by untrusted
devices [2]. These uses of SWIOTLB differ conceptually from historical
use which was a solution to the problem of device addressing
limitations that prevent DMA to some portion of system memory
(typically beyond 4 GiB). IOMMUs also solve the problem of device
addressing limitations and therefore eliminate the need for SWIOTLB for
that purpose. However as mentioned, IOMMUs can use SWIOTLB for other
purposes.

The swiotlb kernel command line parameter does not impact IOMMU related
use of SWIOTLB, and that is intentional. IOMMUs cannot be forced to use
SWIOTLB for all buffers. Update the documentation for the swiotlb
parameter to clarify that SWIOTLB use can only be forced in scenarios
where an IOMMU is not involved.

[1] https://lore.kernel.org/all/20230612153201.554742-16-catalin.marinas@arm.com
[2] https://lore.kernel.org/all/20190906061452.30791-1-baolu.lu@linux.intel.com/
Signed-off-by: T.J. Mercier <tjmercier@google.com>
---
 Documentation/admin-guide/kernel-parameters.txt | 1 +
 Documentation/arch/x86/x86_64/boot-options.rst  | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 213d0719e2b7..84c582ac246c 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -6486,6 +6486,7 @@
 				 to a power of 2.
 			force -- force using of bounce buffers even if they
 			         wouldn't be automatically used by the kernel
+			         where a hardware IOMMU is not involved
 			noforce -- Never use bounce buffers (for debugging)
 
 	switches=	[HW,M68k,EARLY]
diff --git a/Documentation/arch/x86/x86_64/boot-options.rst b/Documentation/arch/x86/x86_64/boot-options.rst
index 137432d34109..066b4bc81583 100644
--- a/Documentation/arch/x86/x86_64/boot-options.rst
+++ b/Documentation/arch/x86/x86_64/boot-options.rst
@@ -285,7 +285,7 @@ iommu options only relevant to the AMD GART hardware IOMMU:
       Always panic when IOMMU overflows.
 
 iommu options only relevant to the software bounce buffering (SWIOTLB) IOMMU
-implementation:
+implementation where a hardware IOMMU is not involved:
 
     swiotlb=<slots>[,force,noforce]
       <slots>
-- 
2.45.0.rc1.225.g2a3ae87e7f-goog
Re: [PATCH] swiotlb: iommu/dma: Clarify swiotlb options apply only to dma-direct
Posted by Petr Tesařík 1 year, 7 months ago
On Fri,  3 May 2024 18:35:26 +0000
"T.J. Mercier" <tjmercier@google.com> wrote:

> IOMMU implementations now sometimes bounce memory through SWIOTLB to
> achieve cacheline alignment [1], or prevent DMA attacks by untrusted
> devices [2]. These uses of SWIOTLB differ conceptually from historical
> use which was a solution to the problem of device addressing
> limitations that prevent DMA to some portion of system memory
> (typically beyond 4 GiB). IOMMUs also solve the problem of device
> addressing limitations and therefore eliminate the need for SWIOTLB for
> that purpose. However as mentioned, IOMMUs can use SWIOTLB for other
> purposes.
> 
> The swiotlb kernel command line parameter does not impact IOMMU related
> use of SWIOTLB, and that is intentional. IOMMUs cannot be forced to use
> SWIOTLB for all buffers. Update the documentation for the swiotlb
> parameter to clarify that SWIOTLB use can only be forced in scenarios
> where an IOMMU is not involved.
> 
> [1] https://lore.kernel.org/all/20230612153201.554742-16-catalin.marinas@arm.com
> [2] https://lore.kernel.org/all/20190906061452.30791-1-baolu.lu@linux.intel.com/
> Signed-off-by: T.J. Mercier <tjmercier@google.com>
> ---
>  Documentation/admin-guide/kernel-parameters.txt | 1 +
>  Documentation/arch/x86/x86_64/boot-options.rst  | 2 +-
>  2 files changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 213d0719e2b7..84c582ac246c 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -6486,6 +6486,7 @@
>  				 to a power of 2.
>  			force -- force using of bounce buffers even if they
>  			         wouldn't be automatically used by the kernel
> +			         where a hardware IOMMU is not involved
>  			noforce -- Never use bounce buffers (for debugging)
>  
>  	switches=	[HW,M68k,EARLY]

Yes, this part is correct. SWIOTLB cannot be forced if there is an IOMMU.

> diff --git a/Documentation/arch/x86/x86_64/boot-options.rst b/Documentation/arch/x86/x86_64/boot-options.rst
> index 137432d34109..066b4bc81583 100644
> --- a/Documentation/arch/x86/x86_64/boot-options.rst
> +++ b/Documentation/arch/x86/x86_64/boot-options.rst
> @@ -285,7 +285,7 @@ iommu options only relevant to the AMD GART hardware IOMMU:
>        Always panic when IOMMU overflows.
>  
>  iommu options only relevant to the software bounce buffering (SWIOTLB) IOMMU
> -implementation:
> +implementation where a hardware IOMMU is not involved:
>  
>      swiotlb=<slots>[,force,noforce]
>        <slots>

But this part needs some improvement. The "swiotlb" option is not
entirely ignored if there is a hardware IOMMU. For example, the size of
the SWIOTLB can be adjusted using "swiotlb=<slots>", and since SWIOTLB
can be used by IOMMUs for other purposes (as you correctly note in the
commit message), this setting is relevant even where a hardware IOMMU
is involved.

Petr T
Re: [PATCH] swiotlb: iommu/dma: Clarify swiotlb options apply only to dma-direct
Posted by T.J. Mercier 1 year, 7 months ago
On Sat, May 4, 2024 at 1:16 AM Petr Tesařík <petr@tesarici.cz> wrote:
>
> On Fri,  3 May 2024 18:35:26 +0000
> "T.J. Mercier" <tjmercier@google.com> wrote:
>
> > IOMMU implementations now sometimes bounce memory through SWIOTLB to
> > achieve cacheline alignment [1], or prevent DMA attacks by untrusted
> > devices [2]. These uses of SWIOTLB differ conceptually from historical
> > use which was a solution to the problem of device addressing
> > limitations that prevent DMA to some portion of system memory
> > (typically beyond 4 GiB). IOMMUs also solve the problem of device
> > addressing limitations and therefore eliminate the need for SWIOTLB for
> > that purpose. However as mentioned, IOMMUs can use SWIOTLB for other
> > purposes.
> >
> > The swiotlb kernel command line parameter does not impact IOMMU related
> > use of SWIOTLB, and that is intentional. IOMMUs cannot be forced to use
> > SWIOTLB for all buffers. Update the documentation for the swiotlb
> > parameter to clarify that SWIOTLB use can only be forced in scenarios
> > where an IOMMU is not involved.
> >
> > [1] https://lore.kernel.org/all/20230612153201.554742-16-catalin.marinas@arm.com
> > [2] https://lore.kernel.org/all/20190906061452.30791-1-baolu.lu@linux.intel.com/
> > Signed-off-by: T.J. Mercier <tjmercier@google.com>
> > ---
> >  Documentation/admin-guide/kernel-parameters.txt | 1 +
> >  Documentation/arch/x86/x86_64/boot-options.rst  | 2 +-
> >  2 files changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > index 213d0719e2b7..84c582ac246c 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -6486,6 +6486,7 @@
> >                                to a power of 2.
> >                       force -- force using of bounce buffers even if they
> >                                wouldn't be automatically used by the kernel
> > +                              where a hardware IOMMU is not involved
> >                       noforce -- Never use bounce buffers (for debugging)
> >
> >       switches=       [HW,M68k,EARLY]
>
> Yes, this part is correct. SWIOTLB cannot be forced if there is an IOMMU.
>
> > diff --git a/Documentation/arch/x86/x86_64/boot-options.rst b/Documentation/arch/x86/x86_64/boot-options.rst
> > index 137432d34109..066b4bc81583 100644
> > --- a/Documentation/arch/x86/x86_64/boot-options.rst
> > +++ b/Documentation/arch/x86/x86_64/boot-options.rst
> > @@ -285,7 +285,7 @@ iommu options only relevant to the AMD GART hardware IOMMU:
> >        Always panic when IOMMU overflows.
> >
> >  iommu options only relevant to the software bounce buffering (SWIOTLB) IOMMU
> > -implementation:
> > +implementation where a hardware IOMMU is not involved:
> >
> >      swiotlb=<slots>[,force,noforce]
> >        <slots>
>
> But this part needs some improvement. The "swiotlb" option is not
> entirely ignored if there is a hardware IOMMU. For example, the size of
> the SWIOTLB can be adjusted using "swiotlb=<slots>", and since SWIOTLB
> can be used by IOMMUs for other purposes (as you correctly note in the
> commit message), this setting is relevant even where a hardware IOMMU
> is involved.
>
> Petr T

Thanks. I think I should also update the commit message:
"The swiotlb=force kernel command line parameter does not impact IOMMU
related use of SWIOTLB"
and title:
"Clarify swiotlb=force option applies only to dma-direct"

As for fixing boot-options.txt, I think it makes the most sense to
expand on just the force option rather than the section summary like
above:
       force
         Force all IO through the software TLB.
+        Hardware IOMMU implementations can use SWIOTLB bounce buffering in
+        some circumstances, but they cannot be forced to always use them, so
+        this option only has an effect when no hardware IOMMU is involved.
       noforce