From: Roman Skakun <roman_skakun@epam.com>

At the moment, Dom0 can't distinguish which devices are protected by
IOMMU and which are not. In some cases, this can cause swiotlb bounce
buffer use for DMA addresses above 32 bits, which in turn can lead
to poor performance. I started a conversation at [1], where we discussed
addition of a new device tree property to mark IOMMU-protected devices for Dom0.

As a result of negotiation at [1], I would like to present two
patches:

1. The first patch for the hypervisor. It adds a new device property
'xen,behind-iommu' to a relevant device node when the device is
IOMMU-protected.

2. The second patch is a Linux kernel counterpart. It detects the said
property and disables swiotlb for a device.

There is a possible issue: some devices may not be able to use DMA
addresses above 32 bit boundaries, so we can have problems in the
direct DMA mechanism when swiotlb-xen is disabled for a such
device. More generally, this can affect any device which DMA address
range is narrower than CPU one.
 
In this case,
the device DMA address should be no bigger than 32 bit boundaries for
each device that is not using swiotlb-xen.

Several ideas on how to overcome it:
1. Do not use high memory for Dom0.
2. Set DMA 32bit mask for each device if swiotlb is not used for this device.
3. Force balloon driver to allocate buffers only below 4GB.

I will be glad to get any comments or suggestions.

[1] https://lore.kernel.org/xen-devel/AM7PR03MB65936E5D0B25567D1B2FAECA85CC9@AM7PR03MB6593.eurprd03.prod.outlook.com/

Roman Skakun (1):
  xen/arm: set iommu property for IOMMU-protected devices

 xen/arch/arm/domain_build.c | 7 +++++++
 1 file changed, 7 insertions(+)

-- 
2.27.0

Hi,

Would you be so kind to look at my patches, it would help me to
know exactly whether I'm moving correctly and whether I've chosen
the right path?
Thanks!

Best regards,
Roman

пн, 4 окт. 2021 г. в 12:54, Roman Skakun <rm.skakun@gmail.com>:
>
> From: Roman Skakun <roman_skakun@epam.com>
>
> At the moment, Dom0 can't distinguish which devices are protected by
> IOMMU and which are not. In some cases, this can cause swiotlb bounce
> buffer use for DMA addresses above 32 bits, which in turn can lead
> to poor performance. I started a conversation at [1], where we discussed
> addition of a new device tree property to mark IOMMU-protected devices for Dom0.
>
> As a result of negotiation at [1], I would like to present two
> patches:
>
> 1. The first patch for the hypervisor. It adds a new device property
> 'xen,behind-iommu' to a relevant device node when the device is
> IOMMU-protected.
>
> 2. The second patch is a Linux kernel counterpart. It detects the said
> property and disables swiotlb for a device.
>
> There is a possible issue: some devices may not be able to use DMA
> addresses above 32 bit boundaries, so we can have problems in the
> direct DMA mechanism when swiotlb-xen is disabled for a such
> device. More generally, this can affect any device which DMA address
> range is narrower than CPU one.
>
> In this case,
> the device DMA address should be no bigger than 32 bit boundaries for
> each device that is not using swiotlb-xen.
>
> Several ideas on how to overcome it:
> 1. Do not use high memory for Dom0.
> 2. Set DMA 32bit mask for each device if swiotlb is not used for this device.
> 3. Force balloon driver to allocate buffers only below 4GB.
>
> I will be glad to get any comments or suggestions.
>
> [1] https://lore.kernel.org/xen-devel/AM7PR03MB65936E5D0B25567D1B2FAECA85CC9@AM7PR03MB6593.eurprd03.prod.outlook.com/
>
> Roman Skakun (1):
>   xen/arm: set iommu property for IOMMU-protected devices
>
>  xen/arch/arm/domain_build.c | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> --
> 2.27.0
>


-- 
Best Regards, Roman.

On 12/10/2021 15:11, Roman Skakun wrote:
> Hi,

Hi Roman,

> Would you be so kind to look at my patches, it would help me to
> know exactly whether I'm moving correctly and whether I've chosen
> the right path?

This is in my queue to review. At the moment, I am prioritizing work 
targeting 4.16. Unfortunately, this RFC was posted after the last 
posting date. So I will review it after the feature freeze.

Best regards,

-- 
Julien Grall

Understood.
Thank you!

вт, 12 окт. 2021 г. в 17:20, Julien Grall <julien@xen.org>:
>
>
>
> On 12/10/2021 15:11, Roman Skakun wrote:
> > Hi,
>
> Hi Roman,
>
> > Would you be so kind to look at my patches, it would help me to
> > know exactly whether I'm moving correctly and whether I've chosen
> > the right path?
>
> This is in my queue to review. At the moment, I am prioritizing work
> targeting 4.16. Unfortunately, this RFC was posted after the last
> posting date. So I will review it after the feature freeze.
>
> Best regards,
>
> --
> Julien Grall



-- 
Best Regards, Roman.