[PATCH 0/2] Memory and PCI definitions for emulated ATS

CLEMENT MATHIEU--DRIF posted 2 patches 2 months, 2 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20250620055620.133027-1-clement.mathieu--drif@eviden.com
Maintainers: "Michael S. Tsirkin" <mst@redhat.com>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, Paolo Bonzini <pbonzini@redhat.com>, Peter Xu <peterx@redhat.com>, David Hildenbrand <david@redhat.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>
include/exec/memattrs.h |  3 +++
include/hw/pci/pci.h    |  9 +++++++++
include/system/memory.h | 23 +++++++++++++++++++++--
3 files changed, 33 insertions(+), 2 deletions(-)
[PATCH 0/2] Memory and PCI definitions for emulated ATS
Posted by CLEMENT MATHIEU--DRIF 2 months, 2 weeks ago
This short series adds the 'address type' bit (concept from PCIe) to the
memory attributes and extends the IOMMUAccessFlags enum. This
will be required to implement ATS support for the virtual IOMMUs.

Address type: Field present in the PCIe R/W requests, it allows devices to
tell the IOMMU if the address provided in the request is physical or not.
In other words, it allows the devices to use a physical address obtained
via ATS and to prevent the IOMMU from trying to remap it on the fly.

Additional IOMMU access flags:
    - Execute Requested
    - Privileged Mode Requested
    - Global
    - Untranslated Only (cannot be used with 'Address type = translated')

Clement Mathieu--Drif (2):
  pci: Add a memory attribute for pre-translated DMA operations
  memory: Add permissions in IOMMUAccessFlags

 include/exec/memattrs.h |  3 +++
 include/hw/pci/pci.h    |  9 +++++++++
 include/system/memory.h | 23 +++++++++++++++++++++--
 3 files changed, 33 insertions(+), 2 deletions(-)

-- 
2.49.0
Re: [PATCH 0/2] Memory and PCI definitions for emulated ATS
Posted by Peter Xu 2 months, 2 weeks ago
On Fri, Jun 20, 2025 at 05:56:49AM +0000, CLEMENT MATHIEU--DRIF wrote:
> This short series adds the 'address type' bit (concept from PCIe) to the
> memory attributes and extends the IOMMUAccessFlags enum. This
> will be required to implement ATS support for the virtual IOMMUs.
> 
> Address type: Field present in the PCIe R/W requests, it allows devices to
> tell the IOMMU if the address provided in the request is physical or not.
> In other words, it allows the devices to use a physical address obtained
> via ATS and to prevent the IOMMU from trying to remap it on the fly.

Two pure questions on the flags, could be relevant to spec:

> 
> Additional IOMMU access flags:
>     - Execute Requested

Does this mean that we can start to put code into DMA regions so that
device can run some day (even if the device may have a core that is totally
different arch v.s. the host's CPUs)?

>     - Privileged Mode Requested
>     - Global
>     - Untranslated Only (cannot be used with 'Address type = translated')

I can understand this with patch 1, but not yet with patch 2.

Patch 1 makes sense to me, IIUC it means the addresses to be used in a pcie
request will be translated addresses which should bypass IOMMU DMAR.

OTOH, patch 2 added it into iotlb access permissions, which I'm not sure
what does it mean.  Perhaps those addresses can only be translated by ATS
pre-translation requests, so that DMA on top of them (in IOVA address
space) will directly fail?

Side note, it might still be more reasonable to put these changes into the
ATS series as the first user of flags.

Thanks,

> 
> Clement Mathieu--Drif (2):
>   pci: Add a memory attribute for pre-translated DMA operations
>   memory: Add permissions in IOMMUAccessFlags
> 
>  include/exec/memattrs.h |  3 +++
>  include/hw/pci/pci.h    |  9 +++++++++
>  include/system/memory.h | 23 +++++++++++++++++++++--
>  3 files changed, 33 insertions(+), 2 deletions(-)
> 
> -- 
> 2.49.0
> 

-- 
Peter Xu
Re: [PATCH 0/2] Memory and PCI definitions for emulated ATS
Posted by CLEMENT MATHIEU--DRIF 2 months, 1 week ago
Hi Peter

On 20/06/2025 4:35 pm, Peter Xu wrote:
> Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe.
> 
> 
> On Fri, Jun 20, 2025 at 05:56:49AM +0000, CLEMENT MATHIEU--DRIF wrote:
>> This short series adds the 'address type' bit (concept from PCIe) to the
>> memory attributes and extends the IOMMUAccessFlags enum. This
>> will be required to implement ATS support for the virtual IOMMUs.
>>
>> Address type: Field present in the PCIe R/W requests, it allows devices to
>> tell the IOMMU if the address provided in the request is physical or not.
>> In other words, it allows the devices to use a physical address obtained
>> via ATS and to prevent the IOMMU from trying to remap it on the fly.
> 
> Two pure questions on the flags, could be relevant to spec:
> 
>>
>> Additional IOMMU access flags:
>>      - Execute Requested
> 
> Does this mean that we can start to put code into DMA regions so that
> device can run some day (even if the device may have a core that is totally
> different arch v.s. the host's 
AFAIU, the spec is so nonrestrictive about this flag that heterogeneous 
arch should not be an issue.

"The definition of what it means for a Function to execute code is 
outside the scope of this specification"

> 
>>      - Privileged Mode Requested
>>      - Global
>>      - Untranslated Only (cannot be used with 'Address type = translated')
> 
> I can understand this with patch 1, but not yet with patch 2.
> 
> Patch 1 makes sense to me, IIUC it means the addresses to be used in a pcie
> request will be translated addresses which should bypass IOMMU DMAR.
> 
> OTOH, patch 2 added it into iotlb access permissions, which I'm not sure
> what does it mean.  Perhaps those addresses can only be translated by ATS
> pre-translation requests, so that DMA on top of them (in IOVA address
> space) will directly fail?

I put this here because the ATS API returns IOMMUTLBEntry structures, 
which contain these flags.

The untranslated-only bit is set in ATS responses to inform the device 
that the requested address cannot be pre-translated and should be 
translated on the fly by the DMA remapping engine. The interrupt range 
commonly falls into this category.

> 
> Side note, it might still be more reasonable to put these changes into the
> ATS series as the first user of flags.

Yes, I can do that.
However, the ATS series will contain ~10/~12 patches, is it a concern?

Thanks
 >cmd

> 
> Thanks,
> 
>>
>> Clement Mathieu--Drif (2):
>>    pci: Add a memory attribute for pre-translated DMA operations
>>    memory: Add permissions in IOMMUAccessFlags
>>
>>   include/exec/memattrs.h |  3 +++
>>   include/hw/pci/pci.h    |  9 +++++++++
>>   include/system/memory.h | 23 +++++++++++++++++++++--
>>   3 files changed, 33 insertions(+), 2 deletions(-)
>>
>> --
>> 2.49.0
>>
> 
> --
> Peter Xu
> 
Re: [PATCH 0/2] Memory and PCI definitions for emulated ATS
Posted by Peter Xu 2 months, 1 week ago
On Mon, Jun 23, 2025 at 05:43:06AM +0000, CLEMENT MATHIEU--DRIF wrote:
> Hi Peter
> 
> On 20/06/2025 4:35 pm, Peter Xu wrote:
> > Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe.
> > 
> > 
> > On Fri, Jun 20, 2025 at 05:56:49AM +0000, CLEMENT MATHIEU--DRIF wrote:
> >> This short series adds the 'address type' bit (concept from PCIe) to the
> >> memory attributes and extends the IOMMUAccessFlags enum. This
> >> will be required to implement ATS support for the virtual IOMMUs.
> >>
> >> Address type: Field present in the PCIe R/W requests, it allows devices to
> >> tell the IOMMU if the address provided in the request is physical or not.
> >> In other words, it allows the devices to use a physical address obtained
> >> via ATS and to prevent the IOMMU from trying to remap it on the fly.
> > 
> > Two pure questions on the flags, could be relevant to spec:
> > 
> >>
> >> Additional IOMMU access flags:
> >>      - Execute Requested
> > 
> > Does this mean that we can start to put code into DMA regions so that
> > device can run some day (even if the device may have a core that is totally
> > different arch v.s. the host's 
> AFAIU, the spec is so nonrestrictive about this flag that heterogeneous 
> arch should not be an issue.
> 
> "The definition of what it means for a Function to execute code is 
> outside the scope of this specification"
> 
> > 
> >>      - Privileged Mode Requested
> >>      - Global
> >>      - Untranslated Only (cannot be used with 'Address type = translated')
> > 
> > I can understand this with patch 1, but not yet with patch 2.
> > 
> > Patch 1 makes sense to me, IIUC it means the addresses to be used in a pcie
> > request will be translated addresses which should bypass IOMMU DMAR.
> > 
> > OTOH, patch 2 added it into iotlb access permissions, which I'm not sure
> > what does it mean.  Perhaps those addresses can only be translated by ATS
> > pre-translation requests, so that DMA on top of them (in IOVA address
> > space) will directly fail?
> 
> I put this here because the ATS API returns IOMMUTLBEntry structures, 
> which contain these flags.
> 
> The untranslated-only bit is set in ATS responses to inform the device 
> that the requested address cannot be pre-translated and should be 
> translated on the fly by the DMA remapping engine. The interrupt range 
> commonly falls into this category.

Ah I see.  Yes this makes sense.

> 
> > 
> > Side note, it might still be more reasonable to put these changes into the
> > ATS series as the first user of flags.
> 
> Yes, I can do that.
> However, the ATS series will contain ~10/~12 patches, is it a concern?

Considering the size of this series, IMHO it should be better to stick with
when they're uesd.

Thanks,

-- 
Peter Xu
Re: [PATCH 0/2] Memory and PCI definitions for emulated ATS
Posted by CLEMENT MATHIEU--DRIF 2 months, 1 week ago

On 23/06/2025 3:15 pm, Peter Xu wrote:
> Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe.
> 
> 
> On Mon, Jun 23, 2025 at 05:43:06AM +0000, CLEMENT MATHIEU--DRIF wrote:
>> Hi Peter
>>
>> On 20/06/2025 4:35 pm, Peter Xu wrote:
>>> Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe.
>>>
>>>
>>> On Fri, Jun 20, 2025 at 05:56:49AM +0000, CLEMENT MATHIEU--DRIF wrote:
>>>> This short series adds the 'address type' bit (concept from PCIe) to the
>>>> memory attributes and extends the IOMMUAccessFlags enum. This
>>>> will be required to implement ATS support for the virtual IOMMUs.
>>>>
>>>> Address type: Field present in the PCIe R/W requests, it allows devices to
>>>> tell the IOMMU if the address provided in the request is physical or not.
>>>> In other words, it allows the devices to use a physical address obtained
>>>> via ATS and to prevent the IOMMU from trying to remap it on the fly.
>>>
>>> Two pure questions on the flags, could be relevant to spec:
>>>
>>>>
>>>> Additional IOMMU access flags:
>>>>       - Execute Requested
>>>
>>> Does this mean that we can start to put code into DMA regions so that
>>> device can run some day (even if the device may have a core that is totally
>>> different arch v.s. the host's
>> AFAIU, the spec is so nonrestrictive about this flag that heterogeneous
>> arch should not be an issue.
>>
>> "The definition of what it means for a Function to execute code is
>> outside the scope of this specification"
>>
>>>
>>>>       - Privileged Mode Requested
>>>>       - Global
>>>>       - Untranslated Only (cannot be used with 'Address type = translated')
>>>
>>> I can understand this with patch 1, but not yet with patch 2.
>>>
>>> Patch 1 makes sense to me, IIUC it means the addresses to be used in a pcie
>>> request will be translated addresses which should bypass IOMMU DMAR.
>>>
>>> OTOH, patch 2 added it into iotlb access permissions, which I'm not sure
>>> what does it mean.  Perhaps those addresses can only be translated by ATS
>>> pre-translation requests, so that DMA on top of them (in IOVA address
>>> space) will directly fail?
>>
>> I put this here because the ATS API returns IOMMUTLBEntry structures,
>> which contain these flags.
>>
>> The untranslated-only bit is set in ATS responses to inform the device
>> that the requested address cannot be pre-translated and should be
>> translated on the fly by the DMA remapping engine. The interrupt range
>> commonly falls into this category.
> 
> Ah I see.  Yes this makes sense.
> 
>>
>>>
>>> Side note, it might still be more reasonable to put these changes into the
>>> ATS series as the first user of flags.
>>
>> Yes, I can do that.
>> However, the ATS series will contain ~10/~12 patches, is it a concern?
> 
> Considering the size of this series, IMHO it should be better to stick with
> when they're uesd.

Ok, will do!
Thanks Peter

 >cmd

> 
> Thanks,
> 
> --
> Peter Xu
>