[Xen-devel] [PATCH] xen/arm: Let the IOMMU be accessible by Dom0 if forcibly disabled in Xen

Oleksandr Tyshchenko posted 1 patch 1 week ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/xen tags/patchew/1565261603-9632-1-git-send-email-olekstysh@gmail.com
xen/arch/arm/domain_build.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)

[Xen-devel] [PATCH] xen/arm: Let the IOMMU be accessible by Dom0 if forcibly disabled in Xen

Posted by Oleksandr Tyshchenko 1 week ago
From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>

Don't skip IOMMU nodes when creating DT for Dom0 if IOMMU has been
forcibly disabled in bootargs (e.g. "iommu=0") in order to let
the IOMMU be accessible by DOM0.

Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
---
I have heard there is a "possible" case when the IOMMU could be accessible by DOM0.
So, I think, for this to work we need to create corresponding DT nodes in the DT
at least.
---
 xen/arch/arm/domain_build.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index d67f7d4..ff88099 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1403,9 +1403,11 @@ static int __init handle_node(struct domain *d, struct kernel_info *kinfo,
 
     /*
      * Even if the IOMMU device is not used by Xen, it should not be
-     * passthrough to DOM0
+     * passthrough to DOM0. The exception here is the fact that IOMMU
+     * has been forcibly disabled in bootargs "iommu=0" in order to let
+     * the IOMMU be accessible by DOM0.
      */
-    if ( device_get_class(node) == DEVICE_IOMMU )
+    if ( device_get_class(node) == DEVICE_IOMMU && iommu_enable )
     {
         dt_dprintk(" IOMMU, skip it\n");
         return 0;
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: Let the IOMMU be accessible by Dom0 if forcibly disabled in Xen

Posted by Roger Pau Monné 1 week ago
On Thu, Aug 08, 2019 at 01:53:23PM +0300, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> Don't skip IOMMU nodes when creating DT for Dom0 if IOMMU has been
> forcibly disabled in bootargs (e.g. "iommu=0") in order to let
> the IOMMU be accessible by DOM0.
> 
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
> I have heard there is a "possible" case when the IOMMU could be accessible by DOM0.
> So, I think, for this to work we need to create corresponding DT nodes in the DT
> at least.

dom0 on ARM being an autotranslated guest I'm not sure how it's going
to program the DMA remapping in the iommu, since it doesn't know the
mfns of the memory it uses at all, hence I don't see the point in
exposing the hardware iommu to dom0 unless there's some emulation done
to make dom0 able to access it.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: Let the IOMMU be accessible by Dom0 if forcibly disabled in Xen

Posted by Oleksandr 1 week ago
On 08.08.19 14:01, Roger Pau Monné wrote:

Hi, Roger.


> On Thu, Aug 08, 2019 at 01:53:23PM +0300, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>
>> Don't skip IOMMU nodes when creating DT for Dom0 if IOMMU has been
>> forcibly disabled in bootargs (e.g. "iommu=0") in order to let
>> the IOMMU be accessible by DOM0.
>>
>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>> ---
>> I have heard there is a "possible" case when the IOMMU could be accessible by DOM0.
>> So, I think, for this to work we need to create corresponding DT nodes in the DT
>> at least.
> dom0 on ARM being an autotranslated guest I'm not sure how it's going
> to program the DMA remapping in the iommu, since it doesn't know the
> mfns of the memory it uses at all, hence I don't see the point in
> exposing the hardware iommu to dom0 unless there's some emulation done
> to make dom0 able to access it.

Currently, Dom0 on ARM is always 1:1 mapped (gfn == mfn). However, I 
don't really know how long this assumption it is going to be true.

Being honest, I don't need this use-case at the moment. I have 
experimented with the code which creates DT nodes for Dom0 a bit, and 
recollected

about the possibility for the IOMMU to be accessible by Dom0. If this 
patch doesn't have any real value, I would be ok to just drop it.

>
> Thanks, Roger.

-- 
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: Let the IOMMU be accessible by Dom0 if forcibly disabled in Xen

Posted by Julien Grall 1 week ago

On 08/08/2019 12:23, Oleksandr wrote:
> 
> On 08.08.19 14:01, Roger Pau Monné wrote:
> 
> Hi, Roger.
> 
> 
>> On Thu, Aug 08, 2019 at 01:53:23PM +0300, Oleksandr Tyshchenko wrote:
>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>
>>> Don't skip IOMMU nodes when creating DT for Dom0 if IOMMU has been
>>> forcibly disabled in bootargs (e.g. "iommu=0") in order to let
>>> the IOMMU be accessible by DOM0.

I don't think your code is doing what you expect... If iommu=0, then Xen will 
not lookup for IOMMUs (iommu_hardware_setup() will not be called). So none of 
the device will have DEVICE_IOMMU set and hence they are already given to dom0.

But I think it is wrong to give the IOMMUs to Dom0 when iommu=0. This is not the 
goal of this option. If you want to passthrough the IOMMU to Dom0, then you 
should use the parameter iommu_hwdom_passthrough.

However, I agree with Roger that giving the IOMMU to dom0 is a pretty bad idea. 
So this should be fixed.

>>>
>>> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>> ---
>>> I have heard there is a "possible" case when the IOMMU could be accessible by 
>>> DOM0.
>>> So, I think, for this to work we need to create corresponding DT nodes in the DT
>>> at least.
>> dom0 on ARM being an autotranslated guest I'm not sure how it's going
>> to program the DMA remapping in the iommu, since it doesn't know the
>> mfns of the memory it uses at all, hence I don't see the point in
>> exposing the hardware iommu to dom0 unless there's some emulation done
>> to make dom0 able to access it.
> 
> Currently, Dom0 on ARM is always 1:1 mapped (gfn == mfn). However, I don't 
> really know how long this assumption it is going to be true.

The 1:1 mapped is only correct for Dom0 RAM. Any foreign mapping will not be 
mapped 1:1.

We actually have code in Linux to keep track of the foreign mapping as any DMA 
access should be using the machine physical address (and not Dom0 physical address).

This brings some headache when IOMMU is used in Xen because we have to add a 1:1 
mapping for foreign page so you can still DMA in it. This will be fun trying to 
fix XSA-300 because of that...

Ideally the 1:1 mapping should only be used when necessary. Unfortunately this 
is not trivial to remove. For a first, Linux is assuming the 1:1 mapping so you 
need to teach Linux to not assume that anymore. So we need to know if the kernel 
is able to deal with it when building dom0.

Furthermore, having an IOMMU on a platform sadly doesn't mean all DMA-capable 
devices will be behind it. This is a bit difficult to find out in Xen.

In short, this is quite a mess to resolve :/.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: Let the IOMMU be accessible by Dom0 if forcibly disabled in Xen

Posted by Oleksandr 1 week ago
Hi, Julien, Roger.



>>
>>
>>> On Thu, Aug 08, 2019 at 01:53:23PM +0300, Oleksandr Tyshchenko wrote:
>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>
>>>> Don't skip IOMMU nodes when creating DT for Dom0 if IOMMU has been
>>>> forcibly disabled in bootargs (e.g. "iommu=0") in order to let
>>>> the IOMMU be accessible by DOM0.
>
> I don't think your code is doing what you expect... If iommu=0, then 
> Xen will not lookup for IOMMUs (iommu_hardware_setup() will not be 
> called). So none of the device will have DEVICE_IOMMU set and hence 
> they are already given to dom0.
>
> But I think it is wrong to give the IOMMUs to Dom0 when iommu=0. This 
> is not the goal of this option. If you want to passthrough the IOMMU 
> to Dom0, then you should use the parameter iommu_hwdom_passthrough.
>
> However, I agree with Roger that giving the IOMMU to dom0 is a pretty 
> bad idea. So this should be fixed.


I fully agree with the arguments provided that it is a bad idea. So, 
please consider that patch as not relevant.


But, I am not sure I follow the last sentence:

 >>> If iommu=0, then Xen will not lookup for IOMMUs 
(iommu_hardware_setup() will not be called). So none of the device will 
have DEVICE_IOMMU set and hence they are already given to dom0.

I can see that devices have DEVICE_IOMMU set. Although, the IOMMU driver 
is not in use, it is present and compatible matches. So, even if 
iommu=0, the IOMMU devices are not given to Dom0, because of skipped. Or 
I missed something?


-- 
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: Let the IOMMU be accessible by Dom0 if forcibly disabled in Xen

Posted by Julien Grall 1 week ago

On 08/08/2019 14:03, Oleksandr wrote:
> 
> Hi, Julien, Roger.

Hi,

> 
> 
> 
>>>
>>>
>>>> On Thu, Aug 08, 2019 at 01:53:23PM +0300, Oleksandr Tyshchenko wrote:
>>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>>
>>>>> Don't skip IOMMU nodes when creating DT for Dom0 if IOMMU has been
>>>>> forcibly disabled in bootargs (e.g. "iommu=0") in order to let
>>>>> the IOMMU be accessible by DOM0.
>>
>> I don't think your code is doing what you expect... If iommu=0, then Xen will 
>> not lookup for IOMMUs (iommu_hardware_setup() will not be called). So none of 
>> the device will have DEVICE_IOMMU set and hence they are already given to dom0.
>>
>> But I think it is wrong to give the IOMMUs to Dom0 when iommu=0. This is not 
>> the goal of this option. If you want to passthrough the IOMMU to Dom0, then 
>> you should use the parameter iommu_hwdom_passthrough.
>>
>> However, I agree with Roger that giving the IOMMU to dom0 is a pretty bad 
>> idea. So this should be fixed.
> 
> 
> I fully agree with the arguments provided that it is a bad idea. So, please 
> consider that patch as not relevant.
> 
> 
> But, I am not sure I follow the last sentence:
> 
>  >>> If iommu=0, then Xen will not lookup for IOMMUs (iommu_hardware_setup() 
> will not be called). So none of the device will have DEVICE_IOMMU set and hence 
> they are already given to dom0.
> 
> I can see that devices have DEVICE_IOMMU set. Although, the IOMMU driver is not 
> in use, it is present and compatible matches. So, even if iommu=0, the IOMMU 
> devices are not given to Dom0, because of skipped. Or I missed something?

I can't see how iommu_hardware_setup() can be called on staging when iommu=0 as 
this is protected by a if ( iommu_enable ).

Can you please give a stack trace how this is called and the version you use? 
WARN() should do it for you.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: Let the IOMMU be accessible by Dom0 if forcibly disabled in Xen

Posted by Oleksandr 1 week ago
Hi


>>
>>
>>
>>>>
>>>>
>>>>> On Thu, Aug 08, 2019 at 01:53:23PM +0300, Oleksandr Tyshchenko wrote:
>>>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>>>
>>>>>> Don't skip IOMMU nodes when creating DT for Dom0 if IOMMU has been
>>>>>> forcibly disabled in bootargs (e.g. "iommu=0") in order to let
>>>>>> the IOMMU be accessible by DOM0.
>>>
>>> I don't think your code is doing what you expect... If iommu=0, then 
>>> Xen will not lookup for IOMMUs (iommu_hardware_setup() will not be 
>>> called). So none of the device will have DEVICE_IOMMU set and hence 
>>> they are already given to dom0.
>>>
>>> But I think it is wrong to give the IOMMUs to Dom0 when iommu=0. 
>>> This is not the goal of this option. If you want to passthrough the 
>>> IOMMU to Dom0, then you should use the parameter 
>>> iommu_hwdom_passthrough.
>>>
>>> However, I agree with Roger that giving the IOMMU to dom0 is a 
>>> pretty bad idea. So this should be fixed.
>>
>>
>> I fully agree with the arguments provided that it is a bad idea. So, 
>> please consider that patch as not relevant.
>>
>>
>> But, I am not sure I follow the last sentence:
>>
>>  >>> If iommu=0, then Xen will not lookup for IOMMUs 
>> (iommu_hardware_setup() will not be called). So none of the device 
>> will have DEVICE_IOMMU set and hence they are already given to dom0.
>>
>> I can see that devices have DEVICE_IOMMU set. Although, the IOMMU 
>> driver is not in use, it is present and compatible matches. So, even 
>> if iommu=0, the IOMMU devices are not given to Dom0, because of 
>> skipped. Or I missed something?
>
> I can't see how iommu_hardware_setup() can be called on staging when 
> iommu=0 as this is protected by a if ( iommu_enable ).
>
> Can you please give a stack trace how this is called and the version 
> you use? WARN() should do it for you.


iommu_hardware_setup() is not called. But, devices have DEVICE_IOMMU 
set, even if "iommu=0". I am based on "7d1460c xen/arm: optee: fix 
compilation with GCC 4.8" + Stefano's reserved-memory series + my IPMMU 
series.


-- 
Regards,

Oleksandr Tyshchenko


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: Let the IOMMU be accessible by Dom0 if forcibly disabled in Xen

Posted by Julien Grall 1 week ago
Hi,

On 08/08/2019 14:36, Oleksandr wrote:
> 
> Hi
> 
> 
>>>
>>>
>>>
>>>>>
>>>>>
>>>>>> On Thu, Aug 08, 2019 at 01:53:23PM +0300, Oleksandr Tyshchenko wrote:
>>>>>>> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
>>>>>>>
>>>>>>> Don't skip IOMMU nodes when creating DT for Dom0 if IOMMU has been
>>>>>>> forcibly disabled in bootargs (e.g. "iommu=0") in order to let
>>>>>>> the IOMMU be accessible by DOM0.
>>>>
>>>> I don't think your code is doing what you expect... If iommu=0, then Xen 
>>>> will not lookup for IOMMUs (iommu_hardware_setup() will not be called). So 
>>>> none of the device will have DEVICE_IOMMU set and hence they are already 
>>>> given to dom0.
>>>>
>>>> But I think it is wrong to give the IOMMUs to Dom0 when iommu=0. This is not 
>>>> the goal of this option. If you want to passthrough the IOMMU to Dom0, then 
>>>> you should use the parameter iommu_hwdom_passthrough.
>>>>
>>>> However, I agree with Roger that giving the IOMMU to dom0 is a pretty bad 
>>>> idea. So this should be fixed.
>>>
>>>
>>> I fully agree with the arguments provided that it is a bad idea. So, please 
>>> consider that patch as not relevant.
>>>
>>>
>>> But, I am not sure I follow the last sentence:
>>>
>>>  >>> If iommu=0, then Xen will not lookup for IOMMUs (iommu_hardware_setup() 
>>> will not be called). So none of the device will have DEVICE_IOMMU set and 
>>> hence they are already given to dom0.
>>>
>>> I can see that devices have DEVICE_IOMMU set. Although, the IOMMU driver is 
>>> not in use, it is present and compatible matches. So, even if iommu=0, the 
>>> IOMMU devices are not given to Dom0, because of skipped. Or I missed something?
>>
>> I can't see how iommu_hardware_setup() can be called on staging when iommu=0 
>> as this is protected by a if ( iommu_enable ).
>>
>> Can you please give a stack trace how this is called and the version you use? 
>> WARN() should do it for you.
> 
> 
> iommu_hardware_setup() is not called. But, devices have DEVICE_IOMMU set, even 
> if "iommu=0". I am based on "7d1460c xen/arm: optee: fix compilation with GCC 
> 4.8" + Stefano's reserved-memory series + my IPMMU series.

Hmmm, what you mean by set is "device_get_class() return DEVICE_IOMMU". This is 
were I got confused and I mixed up with dt_device_set_{protected, used_by}() 
function.

device_get_class() will just lookup for a match and return the class associated. 
So you are right and the node will be skipped in that case. Nothing to modify in 
the current code.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: Let the IOMMU be accessible by Dom0 if forcibly disabled in Xen

Posted by Roger Pau Monné 1 week ago
On Thu, Aug 08, 2019 at 02:23:51PM +0300, Oleksandr wrote:
> 
> On 08.08.19 14:01, Roger Pau Monné wrote:
> 
> Hi, Roger.
> 
> 
> > On Thu, Aug 08, 2019 at 01:53:23PM +0300, Oleksandr Tyshchenko wrote:
> > > From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > > 
> > > Don't skip IOMMU nodes when creating DT for Dom0 if IOMMU has been
> > > forcibly disabled in bootargs (e.g. "iommu=0") in order to let
> > > the IOMMU be accessible by DOM0.
> > > 
> > > Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> > > ---
> > > I have heard there is a "possible" case when the IOMMU could be accessible by DOM0.
> > > So, I think, for this to work we need to create corresponding DT nodes in the DT
> > > at least.
> > dom0 on ARM being an autotranslated guest I'm not sure how it's going
> > to program the DMA remapping in the iommu, since it doesn't know the
> > mfns of the memory it uses at all, hence I don't see the point in
> > exposing the hardware iommu to dom0 unless there's some emulation done
> > to make dom0 able to access it.
> 
> Currently, Dom0 on ARM is always 1:1 mapped (gfn == mfn). However, I don't
> really know how long this assumption it is going to be true.

Yes, I didn't had this in mind when writing the above reply. With
identity mapping in second stage translation it's indeed true that
dom0 might be able to somehow manage an iommu, but I don't think it's
a good idea to rely on this bodge (the identity mappings), and hence I
would advise against exposing the native iommu to dom0.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel