> From: Jason Wang [mailto:jasowang@redhat.com]
> Sent: Friday, November 1, 2019 3:30 PM
> 
> 
> On 2019/10/31 下午10:07, Liu, Yi L wrote:
> >> From: Jason Wang [mailto:jasowang@redhat.com]
> >> Sent: Thursday, October 31, 2019 5:33 AM
> >> Subject: Re: [RFC v2 00/22] intel_iommu: expose Shared Virtual
> Addressing to VM
> >>
> >>
> >> On 2019/10/25 下午6:12, Tian, Kevin wrote:
> >>>> From: Jason Wang [mailto:jasowang@redhat.com]
> >>>> Sent: Friday, October 25, 2019 5:49 PM
> >>>>
> >>>>
> >>>> On 2019/10/24 下午8:34, Liu Yi L wrote:
> >>>>> Shared virtual address (SVA), a.k.a, Shared virtual memory (SVM) on
> >>>>> Intel platforms allow address space sharing between device DMA
> and
> >>>> applications.
> >>>>
> >>>>
> >>>> Interesting, so the below figure demonstrates the case of VM. I
> >>>> wonder how much differences if we compare it with doing SVM
> between
> >>>> device and an ordinary process (e.g dpdk)?
> >>>>
> >>>> Thanks
> >>> One difference is that ordinary process requires only stage-1
> >>> translation, while VM requires nested translation.
> >>
> >> A silly question, then I believe there's no need for VFIO DMA API in this
> case consider
> >> the page table is shared between MMU and IOMMU?
> > Echo Kevin's reply. We use nested translation here. For stage-1, yes, no
> need to use
> > VFIO DMA API. For stage-2, we still use VFIO DMA API to program the
> GPA->HPA
> > mapping to host. :-)
> 
> 
> Cool, two more questions:
> 
> - Can EPT shares its page table with IOMMU L2?

yes, their formats are compatible.

> 
> - Similar to EPT, when GPA->HPA (actually HVA->HPA) is modified by mm,
> VFIO need to use MMU notifier do modify L2 accordingly besides DMA API?
> 

VFIO devices need to pin-down guest memory pages that are mapped
in IOMMU. So notifier is not required since mm won't change the mapping
for those pages.

Thanks
Kevin

On 2019/11/1 下午3:46, Tian, Kevin wrote:
>> From: Jason Wang [mailto:jasowang@redhat.com]
>> Sent: Friday, November 1, 2019 3:30 PM
>>
>>
>> On 2019/10/31 下午10:07, Liu, Yi L wrote:
>>>> From: Jason Wang [mailto:jasowang@redhat.com]
>>>> Sent: Thursday, October 31, 2019 5:33 AM
>>>> Subject: Re: [RFC v2 00/22] intel_iommu: expose Shared Virtual
>> Addressing to VM
>>>>
>>>> On 2019/10/25 下午6:12, Tian, Kevin wrote:
>>>>>> From: Jason Wang [mailto:jasowang@redhat.com]
>>>>>> Sent: Friday, October 25, 2019 5:49 PM
>>>>>>
>>>>>>
>>>>>> On 2019/10/24 下午8:34, Liu Yi L wrote:
>>>>>>> Shared virtual address (SVA), a.k.a, Shared virtual memory (SVM) on
>>>>>>> Intel platforms allow address space sharing between device DMA
>> and
>>>>>> applications.
>>>>>>
>>>>>>
>>>>>> Interesting, so the below figure demonstrates the case of VM. I
>>>>>> wonder how much differences if we compare it with doing SVM
>> between
>>>>>> device and an ordinary process (e.g dpdk)?
>>>>>>
>>>>>> Thanks
>>>>> One difference is that ordinary process requires only stage-1
>>>>> translation, while VM requires nested translation.
>>>> A silly question, then I believe there's no need for VFIO DMA API in this
>> case consider
>>>> the page table is shared between MMU and IOMMU?
>>> Echo Kevin's reply. We use nested translation here. For stage-1, yes, no
>> need to use
>>> VFIO DMA API. For stage-2, we still use VFIO DMA API to program the
>> GPA->HPA
>>> mapping to host. :-)
>>
>> Cool, two more questions:
>>
>> - Can EPT shares its page table with IOMMU L2?
> yes, their formats are compatible.
>
>> - Similar to EPT, when GPA->HPA (actually HVA->HPA) is modified by mm,
>> VFIO need to use MMU notifier do modify L2 accordingly besides DMA API?
>>
> VFIO devices need to pin-down guest memory pages that are mapped
> in IOMMU. So notifier is not required since mm won't change the mapping
> for those pages.


The GUP tends to lead a lot of issues, we may consider to allow 
userspace to choose to not pin them in the future.

Thanks


>
> Thanks
> Kevin

On 2019/11/1 下午4:04, Jason Wang wrote:
>
> On 2019/11/1 下午3:46, Tian, Kevin wrote:
>>> From: Jason Wang [mailto:jasowang@redhat.com]
>>> Sent: Friday, November 1, 2019 3:30 PM
>>>
>>>
>>> On 2019/10/31 下午10:07, Liu, Yi L wrote:
>>>>> From: Jason Wang [mailto:jasowang@redhat.com]
>>>>> Sent: Thursday, October 31, 2019 5:33 AM
>>>>> Subject: Re: [RFC v2 00/22] intel_iommu: expose Shared Virtual
>>> Addressing to VM
>>>>>
>>>>> On 2019/10/25 下午6:12, Tian, Kevin wrote:
>>>>>>> From: Jason Wang [mailto:jasowang@redhat.com]
>>>>>>> Sent: Friday, October 25, 2019 5:49 PM
>>>>>>>
>>>>>>>
>>>>>>> On 2019/10/24 下午8:34, Liu Yi L wrote:
>>>>>>>> Shared virtual address (SVA), a.k.a, Shared virtual memory 
>>>>>>>> (SVM) on
>>>>>>>> Intel platforms allow address space sharing between device DMA
>>> and
>>>>>>> applications.
>>>>>>>
>>>>>>>
>>>>>>> Interesting, so the below figure demonstrates the case of VM. I
>>>>>>> wonder how much differences if we compare it with doing SVM
>>> between
>>>>>>> device and an ordinary process (e.g dpdk)?
>>>>>>>
>>>>>>> Thanks
>>>>>> One difference is that ordinary process requires only stage-1
>>>>>> translation, while VM requires nested translation.
>>>>> A silly question, then I believe there's no need for VFIO DMA API 
>>>>> in this
>>> case consider
>>>>> the page table is shared between MMU and IOMMU?
>>>> Echo Kevin's reply. We use nested translation here. For stage-1, 
>>>> yes, no
>>> need to use
>>>> VFIO DMA API. For stage-2, we still use VFIO DMA API to program the
>>> GPA->HPA
>>>> mapping to host. :-)
>>>
>>> Cool, two more questions:
>>>
>>> - Can EPT shares its page table with IOMMU L2?
>> yes, their formats are compatible.
>>
>>> - Similar to EPT, when GPA->HPA (actually HVA->HPA) is modified by mm,
>>> VFIO need to use MMU notifier do modify L2 accordingly besides DMA API?
>>>
>> VFIO devices need to pin-down guest memory pages that are mapped
>> in IOMMU. So notifier is not required since mm won't change the mapping
>> for those pages.
>
>
> The GUP tends to lead a lot of issues, we may consider to allow 
> userspace to choose to not pin them in the future.


Btw, I'm asking since I see MMU notifier is used by intel-svm.c to flush 
IOTLB. (I don't see any users in kernel source that use that API though 
e.g intel_svm_bind_mm()).

Thanks


>
> Thanks
>
>
>>
>> Thanks
>> Kevin

> From: Jason Wang [mailto:jasowang@redhat.com]
> Sent: Friday, November 1, 2019 4:10 PM
> 
> 
> On 2019/11/1 下午4:04, Jason Wang wrote:
> >
> > On 2019/11/1 下午3:46, Tian, Kevin wrote:
> >>> From: Jason Wang [mailto:jasowang@redhat.com]
> >>> Sent: Friday, November 1, 2019 3:30 PM
> >>>
> >>>
> >>> On 2019/10/31 下午10:07, Liu, Yi L wrote:
> >>>>> From: Jason Wang [mailto:jasowang@redhat.com]
> >>>>> Sent: Thursday, October 31, 2019 5:33 AM
> >>>>> Subject: Re: [RFC v2 00/22] intel_iommu: expose Shared Virtual
> >>> Addressing to VM
> >>>>>
> >>>>> On 2019/10/25 下午6:12, Tian, Kevin wrote:
> >>>>>>> From: Jason Wang [mailto:jasowang@redhat.com]
> >>>>>>> Sent: Friday, October 25, 2019 5:49 PM
> >>>>>>>
> >>>>>>>
> >>>>>>> On 2019/10/24 下午8:34, Liu Yi L wrote:
> >>>>>>>> Shared virtual address (SVA), a.k.a, Shared virtual memory
> >>>>>>>> (SVM) on
> >>>>>>>> Intel platforms allow address space sharing between device DMA
> >>> and
> >>>>>>> applications.
> >>>>>>>
> >>>>>>>
> >>>>>>> Interesting, so the below figure demonstrates the case of VM. I
> >>>>>>> wonder how much differences if we compare it with doing SVM
> >>> between
> >>>>>>> device and an ordinary process (e.g dpdk)?
> >>>>>>>
> >>>>>>> Thanks
> >>>>>> One difference is that ordinary process requires only stage-1
> >>>>>> translation, while VM requires nested translation.
> >>>>> A silly question, then I believe there's no need for VFIO DMA API
> >>>>> in this
> >>> case consider
> >>>>> the page table is shared between MMU and IOMMU?
> >>>> Echo Kevin's reply. We use nested translation here. For stage-1,
> >>>> yes, no
> >>> need to use
> >>>> VFIO DMA API. For stage-2, we still use VFIO DMA API to program the
> >>> GPA->HPA
> >>>> mapping to host. :-)
> >>>
> >>> Cool, two more questions:
> >>>
> >>> - Can EPT shares its page table with IOMMU L2?
> >> yes, their formats are compatible.
> >>
> >>> - Similar to EPT, when GPA->HPA (actually HVA->HPA) is modified by
> mm,
> >>> VFIO need to use MMU notifier do modify L2 accordingly besides DMA
> API?
> >>>
> >> VFIO devices need to pin-down guest memory pages that are mapped
> >> in IOMMU. So notifier is not required since mm won't change the
> mapping
> >> for those pages.
> >
> >
> > The GUP tends to lead a lot of issues, we may consider to allow
> > userspace to choose to not pin them in the future.
> 
> 
> Btw, I'm asking since I see MMU notifier is used by intel-svm.c to flush
> IOTLB. (I don't see any users in kernel source that use that API though
> e.g intel_svm_bind_mm()).
> 

intel-svm.c requires MMU notifier to invalidate IOTLB upon any change
on the CPU page table, when the latter is shared with device in SVA
case. But for VFIO usage, which is based on stage2, the map/unmap 
requests explicitly come from userspace. there is no need to sync with
mm.

Thanks
Kevin