[v1] Add IOMMU support.

[edk2] [RFC] [PATCH 0/3] Add IOMMU support.

Posted by Jiewen Yao 7 years, 7 months ago

This patch series adds IOMMU protocol and updates the consumer
to support IOMMU based DMA access in UEFI.

This patch series can support the BmDmaLib request for AMD SEV.
submitted by Duran, Leo <leo.duran@amd.com> and Brijesh Singh <brijesh.ksingh@gmail.com>.
https://lists.01.org/pipermail/edk2-devel/2017-March/008109.html, and
https://lists.01.org/pipermail/edk2-devel/2017-March/008820.html.
We can have an AMD SEV specific IOMMU driver to produce IOMMU protocol,
and clear SEV in IOMMU->SetAttribute().

This patch series can also support Intel VTd based DMA protection,
requested by Jiewen Yao <jiewen.yao@intel.com>, discussed in
https://lists.01.org/pipermail/edk2-devel/2017-March/008157.html.
We can have an Intel VTd specific IOMMU driver to produce IOMMU protocol,
and update VTd engine to grant or deny access in IOMMU->SetAttribute().

This patch series does not provide a full Intel VTd driver, which
will be provide in other patch in the future.

The purpose of this patch series to review if this IOMMU protocol design
can meet all DMA access and management requirement.

Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Cc: Leo Duran <leo.duran@amd.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>


Jiewen Yao (3):
  MdeModulePkg/Include: Add IOMMU protocol definition.
  MdeModulePkg/PciHostBridge: Add IOMMU support.
  MdeModulePkg/PciBus: Add IOMMU support.

 MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c                    |  12 ++
 MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h                    |  10 ++
 MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf               |   1 +
 MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c                     | 100 ++++++++++++
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c      |   3 +
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridgeDxe.inf |   1 +
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridge.h      |   7 +
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c    | 172 +++++++++++++++++++-
 MdeModulePkg/Include/Protocol/IoMmu.h                      | 132 +++++++++++++++
 MdeModulePkg/MdeModulePkg.dec                              |   3 +
 10 files changed, 436 insertions(+), 5 deletions(-)
 create mode 100644 MdeModulePkg/Include/Protocol/IoMmu.h

-- 
2.7.4.windows.1

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [RFC] [PATCH 0/3] Add IOMMU support.

Posted by Duran, Leo 7 years, 7 months ago

Hi Yao,

This patch-set, in its current form, does not address all of the required SEV functionality for PcIHostBridgeIo.
Basically, we need to intercept all 4 I/O Operations:
- IoMap()
- IoUnmap()
- IoAllocateBuffer()
- IoFreeBuffer()

SEV I/O intercepts would do this:
1) IoMap()
- Allocate an accessible bounce buffer (AllocateType depends on consumer's capabilities).
- Clear SEV mask on retuned mapped DMA buffer
- On DMA Read: CopyMem() from consumer buffer to mapped buffer (bounce operation)

2) IoUnmap()
- On DMA Write: CopyMem() from mapped buffer to consumer buffer (bounce operation)
- Restore SEV mask on mapped DMA buffer

3) IoAllocateBuffer()
- Allocate an accessible buffer (AllocateType depends on consumer's capabilities).
- Clear SEV mask on allocated buffer
- return allocated buffer

4) IoFreeBuffer()
- Restore SEV mask on allocated buffer
- Free allocated buffer

For an sample on how we've intercepted BmDmaLib operations, please refer to the patch-sets posted by Brijesh:
https://lists.01.org/pipermail/edk2-devel/2017-March/008838.html
https://lists.01.org/pipermail/edk2-devel/2017-March/008840.html

Thanks,
Leo

> -----Original Message-----
> From: Jiewen Yao [mailto:jiewen.yao@intel.com]
> Sent: Saturday, March 25, 2017 4:29 AM
> To: edk2-devel@lists.01.org
> Cc: Ruiyu Ni <ruiyu.ni@intel.com>; Duran, Leo <leo.duran@amd.com>;
> Singh, Brijesh <brijesh.singh@amd.com>
> Subject: [RFC] [PATCH 0/3] Add IOMMU support.
> 
> This patch series adds IOMMU protocol and updates the consumer to
> support IOMMU based DMA access in UEFI.
> 
> This patch series can support the BmDmaLib request for AMD SEV.
> submitted by Duran, Leo <leo.duran@amd.com> and Brijesh Singh
> <brijesh.ksingh@gmail.com>.
> https://lists.01.org/pipermail/edk2-devel/2017-March/008109.html, and
> https://lists.01.org/pipermail/edk2-devel/2017-March/008820.html.
> We can have an AMD SEV specific IOMMU driver to produce IOMMU
> protocol, and clear SEV in IOMMU->SetAttribute().
> 
> This patch series can also support Intel VTd based DMA protection,
> requested by Jiewen Yao <jiewen.yao@intel.com>, discussed in
> https://lists.01.org/pipermail/edk2-devel/2017-March/008157.html.
> We can have an Intel VTd specific IOMMU driver to produce IOMMU
> protocol, and update VTd engine to grant or deny access in IOMMU-
> >SetAttribute().
> 
> This patch series does not provide a full Intel VTd driver, which will be
> provide in other patch in the future.
> 
> The purpose of this patch series to review if this IOMMU protocol design can
> meet all DMA access and management requirement.
> 
> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> Cc: Leo Duran <leo.duran@amd.com>
> Cc: Brijesh Singh <brijesh.singh@amd.com>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>
> 
> 
> Jiewen Yao (3):
>   MdeModulePkg/Include: Add IOMMU protocol definition.
>   MdeModulePkg/PciHostBridge: Add IOMMU support.
>   MdeModulePkg/PciBus: Add IOMMU support.
> 
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c                    |  12 ++
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h                    |  10 ++
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf               |   1 +
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c                     | 100 ++++++++++++
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c      |   3 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridgeDxe.inf |   1 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridge.h      |   7 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c    | 172
> +++++++++++++++++++-
>  MdeModulePkg/Include/Protocol/IoMmu.h                      | 132
> +++++++++++++++
>  MdeModulePkg/MdeModulePkg.dec                              |   3 +
>  10 files changed, 436 insertions(+), 5 deletions(-)  create mode 100644
> MdeModulePkg/Include/Protocol/IoMmu.h
> 
> --
> 2.7.4.windows.1

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [RFC] [PATCH 0/3] Add IOMMU support.

Posted by Yao, Jiewen 7 years, 7 months ago

Hi Leo
I do intercept 4 I/O operation in *PciIo*, instead of *PciRootBridgeIo*.

See: https://lists.01.org/pipermail/edk2-devel/2017-March/009021.html

There are 4 gIoMmuProtocol->SetAttribute() I believe you can clear/set SEV action there.

Would you please double check that?

Thank you
Yao Jiewen


From: Duran, Leo [mailto:leo.duran@amd.com]
Sent: Tuesday, March 28, 2017 3:51 AM
To: Yao, Jiewen <jiewen.yao@intel.com>; edk2-devel@lists.01.org
Cc: Ni, Ruiyu <ruiyu.ni@intel.com>; Singh, Brijesh <brijesh.singh@amd.com>
Subject: RE: [RFC] [PATCH 0/3] Add IOMMU support.

Hi Yao,

This patch-set, in its current form, does not address all of the required SEV functionality for PcIHostBridgeIo.
Basically, we need to intercept all 4 I/O Operations:
- IoMap()
- IoUnmap()
- IoAllocateBuffer()
- IoFreeBuffer()

SEV I/O intercepts would do this:
1) IoMap()
- Allocate an accessible bounce buffer (AllocateType depends on consumer's capabilities).
- Clear SEV mask on retuned mapped DMA buffer
- On DMA Read: CopyMem() from consumer buffer to mapped buffer (bounce operation)

2) IoUnmap()
- On DMA Write: CopyMem() from mapped buffer to consumer buffer (bounce operation)
- Restore SEV mask on mapped DMA buffer

3) IoAllocateBuffer()
- Allocate an accessible buffer (AllocateType depends on consumer's capabilities).
- Clear SEV mask on allocated buffer
- return allocated buffer

4) IoFreeBuffer()
- Restore SEV mask on allocated buffer
- Free allocated buffer

For an sample on how we've intercepted BmDmaLib operations, please refer to the patch-sets posted by Brijesh:
https://lists.01.org/pipermail/edk2-devel/2017-March/008838.html
https://lists.01.org/pipermail/edk2-devel/2017-March/008840.html

Thanks,
Leo

> -----Original Message-----
> From: Jiewen Yao [mailto:jiewen.yao@intel.com]
> Sent: Saturday, March 25, 2017 4:29 AM
> To: edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com>>; Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com>>;
> Singh, Brijesh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>
> Subject: [RFC] [PATCH 0/3] Add IOMMU support.
>
> This patch series adds IOMMU protocol and updates the consumer to
> support IOMMU based DMA access in UEFI.
>
> This patch series can support the BmDmaLib request for AMD SEV.
> submitted by Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com>> and Brijesh Singh
> <brijesh.ksingh@gmail.com<mailto:brijesh.ksingh@gmail.com>>.
> https://lists.01.org/pipermail/edk2-devel/2017-March/008109.html, and
> https://lists.01.org/pipermail/edk2-devel/2017-March/008820.html.
> We can have an AMD SEV specific IOMMU driver to produce IOMMU
> protocol, and clear SEV in IOMMU->SetAttribute().
>
> This patch series can also support Intel VTd based DMA protection,
> requested by Jiewen Yao <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>, discussed in
> https://lists.01.org/pipermail/edk2-devel/2017-March/008157.html.
> We can have an Intel VTd specific IOMMU driver to produce IOMMU
> protocol, and update VTd engine to grant or deny access in IOMMU-
> >SetAttribute().
>
> This patch series does not provide a full Intel VTd driver, which will be
> provide in other patch in the future.
>
> The purpose of this patch series to review if this IOMMU protocol design can
> meet all DMA access and management requirement.
>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com>>
> Cc: Leo Duran <leo.duran@amd.com<mailto:leo.duran@amd.com>>
> Cc: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Jiewen Yao <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>
>
>
> Jiewen Yao (3):
>   MdeModulePkg/Include: Add IOMMU protocol definition.
>   MdeModulePkg/PciHostBridge: Add IOMMU support.
>   MdeModulePkg/PciBus: Add IOMMU support.
>
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c                    |  12 ++
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h                    |  10 ++
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf               |   1 +
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c                     | 100 ++++++++++++
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c      |   3 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridgeDxe.inf |   1 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridge.h      |   7 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c    | 172
> +++++++++++++++++++-
>  MdeModulePkg/Include/Protocol/IoMmu.h                      | 132
> +++++++++++++++
>  MdeModulePkg/MdeModulePkg.dec                              |   3 +
>  10 files changed, 436 insertions(+), 5 deletions(-)  create mode 100644
> MdeModulePkg/Include/Protocol/IoMmu.h
>
> --
> 2.7.4.windows.1
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [RFC] [PATCH 0/3] Add IOMMU support.

Posted by Yao, Jiewen 7 years, 7 months ago

I forget mentioning that I have to do it in PciIo instead of PciRootBridgeIo.
The reason is that Intel VTd solution need know which PCI device submit the DMA access request, so that it can only modify the IOMMU paging for this specific device.

I believe AMD IOMMU has similar requirement.

However, this info is not needed, if you just want to enable AMD SEV solution. In that case, you can ignore the DeviceHandle, and just update SEV clear/set bit for that memory.


BTW: I just realize I miss one thing. I call gIoMmuProtocol->SetAttribute, after it is returned from PciRootBridgeIo.

For AMD SEC, I think we need do CopyMem again, if this is DMA read request. I will update my patch for that.


Thank you
Yao Jiewen

From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Yao, Jiewen
Sent: Tuesday, March 28, 2017 9:41 AM
To: Duran, Leo <leo.duran@amd.com>; edk2-devel@lists.01.org
Cc: Ni, Ruiyu <ruiyu.ni@intel.com>; Singh, Brijesh <brijesh.singh@amd.com>
Subject: Re: [edk2] [RFC] [PATCH 0/3] Add IOMMU support.

Hi Leo
I do intercept 4 I/O operation in *PciIo*, instead of *PciRootBridgeIo*.

See: https://lists.01.org/pipermail/edk2-devel/2017-March/009021.html

There are 4 gIoMmuProtocol->SetAttribute() I believe you can clear/set SEV action there.

Would you please double check that?

Thank you
Yao Jiewen


From: Duran, Leo [mailto:leo.duran@amd.com]
Sent: Tuesday, March 28, 2017 3:51 AM
To: Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
Cc: Ni, Ruiyu <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com>>; Singh, Brijesh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>
Subject: RE: [RFC] [PATCH 0/3] Add IOMMU support.

Hi Yao,

This patch-set, in its current form, does not address all of the required SEV functionality for PcIHostBridgeIo.
Basically, we need to intercept all 4 I/O Operations:
- IoMap()
- IoUnmap()
- IoAllocateBuffer()
- IoFreeBuffer()

SEV I/O intercepts would do this:
1) IoMap()
- Allocate an accessible bounce buffer (AllocateType depends on consumer's capabilities).
- Clear SEV mask on retuned mapped DMA buffer
- On DMA Read: CopyMem() from consumer buffer to mapped buffer (bounce operation)

2) IoUnmap()
- On DMA Write: CopyMem() from mapped buffer to consumer buffer (bounce operation)
- Restore SEV mask on mapped DMA buffer

3) IoAllocateBuffer()
- Allocate an accessible buffer (AllocateType depends on consumer's capabilities).
- Clear SEV mask on allocated buffer
- return allocated buffer

4) IoFreeBuffer()
- Restore SEV mask on allocated buffer
- Free allocated buffer

For an sample on how we've intercepted BmDmaLib operations, please refer to the patch-sets posted by Brijesh:
https://lists.01.org/pipermail/edk2-devel/2017-March/008838.html
https://lists.01.org/pipermail/edk2-devel/2017-March/008840.html

Thanks,
Leo

> -----Original Message-----
> From: Jiewen Yao [mailto:jiewen.yao@intel.com]
> Sent: Saturday, March 25, 2017 4:29 AM
> To: edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org>>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com%3cmailto:ruiyu.ni@intel.com>>>; Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>>;
> Singh, Brijesh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com<mailto:brijesh.singh@amd.com%3cmailto:brijesh.singh@amd.com>>>
> Subject: [RFC] [PATCH 0/3] Add IOMMU support.
>
> This patch series adds IOMMU protocol and updates the consumer to
> support IOMMU based DMA access in UEFI.
>
> This patch series can support the BmDmaLib request for AMD SEV.
> submitted by Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>> and Brijesh Singh
> <brijesh.ksingh@gmail.com<mailto:brijesh.ksingh@gmail.com<mailto:brijesh.ksingh@gmail.com%3cmailto:brijesh.ksingh@gmail.com>>>.
> https://lists.01.org/pipermail/edk2-devel/2017-March/008109.html, and
> https://lists.01.org/pipermail/edk2-devel/2017-March/008820.html.
> We can have an AMD SEV specific IOMMU driver to produce IOMMU
> protocol, and clear SEV in IOMMU->SetAttribute().
>
> This patch series can also support Intel VTd based DMA protection,
> requested by Jiewen Yao <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com<mailto:jiewen.yao@intel.com%3cmailto:jiewen.yao@intel.com>>>, discussed in
> https://lists.01.org/pipermail/edk2-devel/2017-March/008157.html.
> We can have an Intel VTd specific IOMMU driver to produce IOMMU
> protocol, and update VTd engine to grant or deny access in IOMMU-
> >SetAttribute().
>
> This patch series does not provide a full Intel VTd driver, which will be
> provide in other patch in the future.
>
> The purpose of this patch series to review if this IOMMU protocol design can
> meet all DMA access and management requirement.
>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com%3cmailto:ruiyu.ni@intel.com>>>
> Cc: Leo Duran <leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>>
> Cc: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com<mailto:brijesh.singh@amd.com%3cmailto:brijesh.singh@amd.com>>>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Jiewen Yao <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com<mailto:jiewen.yao@intel.com%3cmailto:jiewen.yao@intel.com>>>
>
>
> Jiewen Yao (3):
>   MdeModulePkg/Include: Add IOMMU protocol definition.
>   MdeModulePkg/PciHostBridge: Add IOMMU support.
>   MdeModulePkg/PciBus: Add IOMMU support.
>
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c                    |  12 ++
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h                    |  10 ++
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf               |   1 +
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c                     | 100 ++++++++++++
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c      |   3 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridgeDxe.inf |   1 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridge.h      |   7 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c    | 172
> +++++++++++++++++++-
>  MdeModulePkg/Include/Protocol/IoMmu.h                      | 132
> +++++++++++++++
>  MdeModulePkg/MdeModulePkg.dec                              |   3 +
>  10 files changed, 436 insertions(+), 5 deletions(-)  create mode 100644
> MdeModulePkg/Include/Protocol/IoMmu.h
>
> --
> 2.7.4.windows.1
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
https://lists.01.org/mailman/listinfo/edk2-devel
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [RFC] [PATCH 0/3] Add IOMMU support.

Posted by Ni, Ruiyu 7 years, 7 months ago

Jiewen,
For performance consideration, do you think we may define a bit as a hint to tell PciHostBridge (or PciBus, I don't know) whether a post-CopyMem is needed?

Thanks/Ray

From: Yao, Jiewen
Sent: Tuesday, March 28, 2017 9:55 AM
To: Yao, Jiewen <jiewen.yao@intel.com>; Duran, Leo <leo.duran@amd.com>; edk2-devel@lists.01.org
Cc: Ni, Ruiyu <ruiyu.ni@intel.com>; Singh, Brijesh <brijesh.singh@amd.com>
Subject: RE: [RFC] [PATCH 0/3] Add IOMMU support.

I forget mentioning that I have to do it in PciIo instead of PciRootBridgeIo.
The reason is that Intel VTd solution need know which PCI device submit the DMA access request, so that it can only modify the IOMMU paging for this specific device.

I believe AMD IOMMU has similar requirement.

However, this info is not needed, if you just want to enable AMD SEV solution. In that case, you can ignore the DeviceHandle, and just update SEV clear/set bit for that memory.


BTW: I just realize I miss one thing. I call gIoMmuProtocol->SetAttribute, after it is returned from PciRootBridgeIo.

For AMD SEC, I think we need do CopyMem again, if this is DMA read request. I will update my patch for that.


Thank you
Yao Jiewen

From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Yao, Jiewen
Sent: Tuesday, March 28, 2017 9:41 AM
To: Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
Cc: Ni, Ruiyu <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com>>; Singh, Brijesh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>
Subject: Re: [edk2] [RFC] [PATCH 0/3] Add IOMMU support.

Hi Leo
I do intercept 4 I/O operation in *PciIo*, instead of *PciRootBridgeIo*.

See: https://lists.01.org/pipermail/edk2-devel/2017-March/009021.html

There are 4 gIoMmuProtocol->SetAttribute() I believe you can clear/set SEV action there.

Would you please double check that?

Thank you
Yao Jiewen


From: Duran, Leo [mailto:leo.duran@amd.com]
Sent: Tuesday, March 28, 2017 3:51 AM
To: Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
Cc: Ni, Ruiyu <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com>>; Singh, Brijesh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>
Subject: RE: [RFC] [PATCH 0/3] Add IOMMU support.

Hi Yao,

This patch-set, in its current form, does not address all of the required SEV functionality for PcIHostBridgeIo.
Basically, we need to intercept all 4 I/O Operations:
- IoMap()
- IoUnmap()
- IoAllocateBuffer()
- IoFreeBuffer()

SEV I/O intercepts would do this:
1) IoMap()
- Allocate an accessible bounce buffer (AllocateType depends on consumer's capabilities).
- Clear SEV mask on retuned mapped DMA buffer
- On DMA Read: CopyMem() from consumer buffer to mapped buffer (bounce operation)

2) IoUnmap()
- On DMA Write: CopyMem() from mapped buffer to consumer buffer (bounce operation)
- Restore SEV mask on mapped DMA buffer

3) IoAllocateBuffer()
- Allocate an accessible buffer (AllocateType depends on consumer's capabilities).
- Clear SEV mask on allocated buffer
- return allocated buffer

4) IoFreeBuffer()
- Restore SEV mask on allocated buffer
- Free allocated buffer

For an sample on how we've intercepted BmDmaLib operations, please refer to the patch-sets posted by Brijesh:
https://lists.01.org/pipermail/edk2-devel/2017-March/008838.html
https://lists.01.org/pipermail/edk2-devel/2017-March/008840.html

Thanks,
Leo

> -----Original Message-----
> From: Jiewen Yao [mailto:jiewen.yao@intel.com]
> Sent: Saturday, March 25, 2017 4:29 AM
> To: edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org>>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com%3cmailto:ruiyu.ni@intel.com>>>; Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>>;
> Singh, Brijesh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com<mailto:brijesh.singh@amd.com%3cmailto:brijesh.singh@amd.com>>>
> Subject: [RFC] [PATCH 0/3] Add IOMMU support.
>
> This patch series adds IOMMU protocol and updates the consumer to
> support IOMMU based DMA access in UEFI.
>
> This patch series can support the BmDmaLib request for AMD SEV.
> submitted by Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>> and Brijesh Singh
> <brijesh.ksingh@gmail.com<mailto:brijesh.ksingh@gmail.com<mailto:brijesh.ksingh@gmail.com%3cmailto:brijesh.ksingh@gmail.com>>>.
> https://lists.01.org/pipermail/edk2-devel/2017-March/008109.html, and
> https://lists.01.org/pipermail/edk2-devel/2017-March/008820.html.
> We can have an AMD SEV specific IOMMU driver to produce IOMMU
> protocol, and clear SEV in IOMMU->SetAttribute().
>
> This patch series can also support Intel VTd based DMA protection,
> requested by Jiewen Yao <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com<mailto:jiewen.yao@intel.com%3cmailto:jiewen.yao@intel.com>>>, discussed in
> https://lists.01.org/pipermail/edk2-devel/2017-March/008157.html.
> We can have an Intel VTd specific IOMMU driver to produce IOMMU
> protocol, and update VTd engine to grant or deny access in IOMMU-
> >SetAttribute().
>
> This patch series does not provide a full Intel VTd driver, which will be
> provide in other patch in the future.
>
> The purpose of this patch series to review if this IOMMU protocol design can
> meet all DMA access and management requirement.
>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com%3cmailto:ruiyu.ni@intel.com>>>
> Cc: Leo Duran <leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>>
> Cc: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com<mailto:brijesh.singh@amd.com%3cmailto:brijesh.singh@amd.com>>>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Jiewen Yao <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com<mailto:jiewen.yao@intel.com%3cmailto:jiewen.yao@intel.com>>>
>
>
> Jiewen Yao (3):
>   MdeModulePkg/Include: Add IOMMU protocol definition.
>   MdeModulePkg/PciHostBridge: Add IOMMU support.
>   MdeModulePkg/PciBus: Add IOMMU support.
>
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c                    |  12 ++
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h                    |  10 ++
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf               |   1 +
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c                     | 100 ++++++++++++
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c      |   3 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridgeDxe.inf |   1 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridge.h      |   7 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c    | 172
> +++++++++++++++++++-
>  MdeModulePkg/Include/Protocol/IoMmu.h                      | 132
> +++++++++++++++
>  MdeModulePkg/MdeModulePkg.dec                              |   3 +
>  10 files changed, 436 insertions(+), 5 deletions(-)  create mode 100644
> MdeModulePkg/Include/Protocol/IoMmu.h
>
> --
> 2.7.4.windows.1
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
https://lists.01.org/mailman/listinfo/edk2-devel
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: [edk2] [RFC] [PATCH 0/3] Add IOMMU support.

Posted by Yao, Jiewen 7 years, 7 months ago

I think IOMMU protocol can be the hint.

From: Ni, Ruiyu
Sent: Tuesday, March 28, 2017 10:25 AM
To: Yao, Jiewen <jiewen.yao@intel.com>; Duran, Leo <leo.duran@amd.com>; edk2-devel@lists.01.org
Cc: Singh, Brijesh <brijesh.singh@amd.com>
Subject: RE: [RFC] [PATCH 0/3] Add IOMMU support.

Jiewen,
For performance consideration, do you think we may define a bit as a hint to tell PciHostBridge (or PciBus, I don't know) whether a post-CopyMem is needed?

Thanks/Ray

From: Yao, Jiewen
Sent: Tuesday, March 28, 2017 9:55 AM
To: Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>; Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
Cc: Ni, Ruiyu <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com>>; Singh, Brijesh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>
Subject: RE: [RFC] [PATCH 0/3] Add IOMMU support.

I forget mentioning that I have to do it in PciIo instead of PciRootBridgeIo.
The reason is that Intel VTd solution need know which PCI device submit the DMA access request, so that it can only modify the IOMMU paging for this specific device.

I believe AMD IOMMU has similar requirement.

However, this info is not needed, if you just want to enable AMD SEV solution. In that case, you can ignore the DeviceHandle, and just update SEV clear/set bit for that memory.

BTW: I just realize I miss one thing. I call gIoMmuProtocol->SetAttribute, after it is returned from PciRootBridgeIo.

For AMD SEC, I think we need do CopyMem again, if this is DMA read request. I will update my patch for that.

Thank you
Yao Jiewen

From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Yao, Jiewen
Sent: Tuesday, March 28, 2017 9:41 AM
To: Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
Cc: Ni, Ruiyu <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com>>; Singh, Brijesh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>
Subject: Re: [edk2] [RFC] [PATCH 0/3] Add IOMMU support.

Hi Leo
I do intercept 4 I/O operation in *PciIo*, instead of *PciRootBridgeIo*.

See: https://lists.01.org/pipermail/edk2-devel/2017-March/009021.html

There are 4 gIoMmuProtocol->SetAttribute() I believe you can clear/set SEV action there.

Would you please double check that?

Thank you
Yao Jiewen

From: Duran, Leo [mailto:leo.duran@amd.com]
Sent: Tuesday, March 28, 2017 3:51 AM
To: Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>>; edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
Cc: Ni, Ruiyu <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com>>; Singh, Brijesh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com>>
Subject: RE: [RFC] [PATCH 0/3] Add IOMMU support.

Hi Yao,

This patch-set, in its current form, does not address all of the required SEV functionality for PcIHostBridgeIo.
Basically, we need to intercept all 4 I/O Operations:
- IoMap()
- IoUnmap()
- IoAllocateBuffer()
- IoFreeBuffer()

SEV I/O intercepts would do this:
1) IoMap()
- Allocate an accessible bounce buffer (AllocateType depends on consumer's capabilities).
- Clear SEV mask on retuned mapped DMA buffer
- On DMA Read: CopyMem() from consumer buffer to mapped buffer (bounce operation)

2) IoUnmap()
- On DMA Write: CopyMem() from mapped buffer to consumer buffer (bounce operation)
- Restore SEV mask on mapped DMA buffer

3) IoAllocateBuffer()
- Allocate an accessible buffer (AllocateType depends on consumer's capabilities).
- Clear SEV mask on allocated buffer
- return allocated buffer

4) IoFreeBuffer()
- Restore SEV mask on allocated buffer
- Free allocated buffer

For an sample on how we've intercepted BmDmaLib operations, please refer to the patch-sets posted by Brijesh:
https://lists.01.org/pipermail/edk2-devel/2017-March/008838.html
https://lists.01.org/pipermail/edk2-devel/2017-March/008840.html

Thanks,
Leo

> -----Original Message-----
> From: Jiewen Yao [mailto:jiewen.yao@intel.com]
> Sent: Saturday, March 25, 2017 4:29 AM
> To: edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org>>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com%3cmailto:ruiyu.ni@intel.com>>>; Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>>;
> Singh, Brijesh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com<mailto:brijesh.singh@amd.com%3cmailto:brijesh.singh@amd.com>>>
> Subject: [RFC] [PATCH 0/3] Add IOMMU support.
>
> This patch series adds IOMMU protocol and updates the consumer to
> support IOMMU based DMA access in UEFI.
>
> This patch series can support the BmDmaLib request for AMD SEV.
> submitted by Duran, Leo <leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>> and Brijesh Singh
> <brijesh.ksingh@gmail.com<mailto:brijesh.ksingh@gmail.com<mailto:brijesh.ksingh@gmail.com%3cmailto:brijesh.ksingh@gmail.com>>>.
> https://lists.01.org/pipermail/edk2-devel/2017-March/008109.html, and
> https://lists.01.org/pipermail/edk2-devel/2017-March/008820.html.
> We can have an AMD SEV specific IOMMU driver to produce IOMMU
> protocol, and clear SEV in IOMMU->SetAttribute().
>
> This patch series can also support Intel VTd based DMA protection,
> requested by Jiewen Yao <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com<mailto:jiewen.yao@intel.com%3cmailto:jiewen.yao@intel.com>>>, discussed in
> https://lists.01.org/pipermail/edk2-devel/2017-March/008157.html.
> We can have an Intel VTd specific IOMMU driver to produce IOMMU
> protocol, and update VTd engine to grant or deny access in IOMMU-
> >SetAttribute().
>
> This patch series does not provide a full Intel VTd driver, which will be
> provide in other patch in the future.
>
> The purpose of this patch series to review if this IOMMU protocol design can
> meet all DMA access and management requirement.
>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com<mailto:ruiyu.ni@intel.com%3cmailto:ruiyu.ni@intel.com>>>
> Cc: Leo Duran <leo.duran@amd.com<mailto:leo.duran@amd.com<mailto:leo.duran@amd.com%3cmailto:leo.duran@amd.com>>>
> Cc: Brijesh Singh <brijesh.singh@amd.com<mailto:brijesh.singh@amd.com<mailto:brijesh.singh@amd.com%3cmailto:brijesh.singh@amd.com>>>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Jiewen Yao <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com<mailto:jiewen.yao@intel.com%3cmailto:jiewen.yao@intel.com>>>
>
>
> Jiewen Yao (3):
>   MdeModulePkg/Include: Add IOMMU protocol definition.
>   MdeModulePkg/PciHostBridge: Add IOMMU support.
>   MdeModulePkg/PciBus: Add IOMMU support.
>
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c                    |  12 ++
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h                    |  10 ++
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf               |   1 +
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c                     | 100 ++++++++++++
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c      |   3 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridgeDxe.inf |   1 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridge.h      |   7 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c    | 172
> +++++++++++++++++++-
>  MdeModulePkg/Include/Protocol/IoMmu.h                      | 132
> +++++++++++++++
>  MdeModulePkg/MdeModulePkg.dec                              |   3 +
>  10 files changed, 436 insertions(+), 5 deletions(-)  create mode 100644
> MdeModulePkg/Include/Protocol/IoMmu.h
>
> --
> 2.7.4.windows.1
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>
https://lists.01.org/mailman/listinfo/edk2-devel
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel