[edk2-devel] [PATCH 00/14] Introduce Lazy-accept for Tdx guest

Min Xu posted 14 patches 2 years, 5 months ago
Failed in applying to current master (apply log)
There is a newer version of this series
MdeModulePkg/Core/Dxe/DxeMain.inf             |   1 +
MdeModulePkg/Core/Dxe/Gcd/Gcd.c               |   5 +
MdeModulePkg/Core/Dxe/Mem/Imem.h              |  16 ++
MdeModulePkg/Core/Dxe/Mem/Page.c              | 218 ++++++++++++++++++
MdeModulePkg/Core/Dxe/Mem/Pool.c              |  14 ++
MdeModulePkg/Include/Pi/PrePiHob.h            |  20 ++
MdePkg/Include/Pi/PiDxeCis.h                  |   5 +
MdePkg/Include/Pi/PiHob.h                     |  11 +-
MdePkg/Include/Protocol/MemoryAccept.h        |  37 +++
MdePkg/Include/Uefi/UefiMultiPhase.h          |   5 +
MdePkg/MdePkg.dec                             |   3 +
OvmfPkg/Include/Library/PlatformInitLib.h     |   6 +
OvmfPkg/Include/WorkArea.h                    |   1 +
OvmfPkg/IntelTdx/IntelTdxX64.dsc              |   8 +
.../PrePiHobListPointer.c                     |   4 +-
.../PrePiHobListPointerLibTdx.inf             |   2 +-
OvmfPkg/Library/PeilessStartupLib/Hob.c       |  26 ++-
.../PeilessStartupLib/PeilessStartupLib.inf   |   1 +
OvmfPkg/Library/PlatformInitLib/IntelTdx.c    | 145 +++++++++++-
OvmfPkg/Library/PlatformInitLib/MemDetect.c   |  27 +++
.../PlatformInitLib/PlatformInitLib.inf       |   1 +
OvmfPkg/OvmfPkg.dec                           |   4 +
OvmfPkg/OvmfPkgX64.dsc                        |   9 +
OvmfPkg/PlatformPei/MemDetect.c               |   5 +
OvmfPkg/TdxDxe/TdxDxe.c                       | 103 +++++++++
OvmfPkg/TdxDxe/TdxDxe.inf                     |   2 +
.../UefiShellDebug1CommandsLib/MemMap.c       |   6 +
27 files changed, 666 insertions(+), 19 deletions(-)
create mode 100644 MdeModulePkg/Include/Pi/PrePiHob.h
create mode 100644 MdePkg/Include/Protocol/MemoryAccept.h
[edk2-devel] [PATCH 00/14] Introduce Lazy-accept for Tdx guest
Posted by Min Xu 2 years, 5 months ago
RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937

UnacceptedMemory is one of the four defined types of TD memory in Intel
TDX guest. TDVF must invoke TDCALL [TDG.MEM.PAGE.ACCEPT] the unaccepted
memory before use it. See [TDVF] Section 7.1.
TDVF: https://www.intel.com/content/dam/develop/external/us/en/
documents/tdx-virtual-firmware-design-guide-rev-1.01.pdf 

It is a time-consuming task which impacts the boot performance badly.
One of the mitigation is the lazy-accept mechanism. That the whole system
memory is divided into 2 parts, one is accepted in bios phase, the other
is tagged as EfiGcdMemoryTypeUnaccepted and OS will handle these
"unaccepted" memories.
See "UEFI Spec v2.9 Table 7-5 Memory Type Usage before ExitBootServices()"

Patch 1-4: 
  Introduce lazy-accept related definitions.

Patch 5-6:
  Update Dxe and shell for unaccepted memory.

Patch 7 - 11:
  Update OvmfPkg for unaccepted memory.

Patch 12 - 13:
  Introduce EfiMemoryAcceptProtocol and realize it in TdxDxe.

Patch 14:
  Update Pool and Page functions to accept memory when OOM occurs.

Code: https://github.com/mxu9/edk2/tree/lazyaccept.v1

Cc: Zhichao Gao <zhichao.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Zhiguang Liu <zhiguang.liu@intel.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>

Jiaqi Gao (2):
  MdePkg: The prototype definition of EfiMemoryAcceptProtocol
  MdeModulePkg: Pool and page functions accept memory when OOM occurs

Min M Xu (12):
  MdeModulePkg: Add PrePiHob.h
  MdePkg: Increase EFI_RESOURCE_MAX_MEMORY_TYPE
  OvmfPkg: Use EFI_RESOURCE_MEMORY_UNACCEPTED which defined in
    MdeModulePkg
  MdePkg: Add UEFI Unaccepted memory definition
  MdeModulePkg: Update Dxe to handle unaccepted memory type
  ShellPkg: Update shell command memmap to show unaccepted memory
  OvmfPkg: Add PCD and DEFINEs for Lazy Accept page.
  OvmfPkg: Use PcdOvmfWorkAreaBase instead of PcdSevEsWorkAreaBase
  OvmfPkg: Add MaxAcceptedMemoryAddress in TDX work area
  OvmfPkg: Introduce lazy accept in PlatformInitLib and PlatformPei
  OvmfPkg: Update ConstructFwHobList for lazy accept
  OvmfPkg: Realize EfiMemoryAcceptProtocol in TdxDxe

 MdeModulePkg/Core/Dxe/DxeMain.inf             |   1 +
 MdeModulePkg/Core/Dxe/Gcd/Gcd.c               |   5 +
 MdeModulePkg/Core/Dxe/Mem/Imem.h              |  16 ++
 MdeModulePkg/Core/Dxe/Mem/Page.c              | 218 ++++++++++++++++++
 MdeModulePkg/Core/Dxe/Mem/Pool.c              |  14 ++
 MdeModulePkg/Include/Pi/PrePiHob.h            |  20 ++
 MdePkg/Include/Pi/PiDxeCis.h                  |   5 +
 MdePkg/Include/Pi/PiHob.h                     |  11 +-
 MdePkg/Include/Protocol/MemoryAccept.h        |  37 +++
 MdePkg/Include/Uefi/UefiMultiPhase.h          |   5 +
 MdePkg/MdePkg.dec                             |   3 +
 OvmfPkg/Include/Library/PlatformInitLib.h     |   6 +
 OvmfPkg/Include/WorkArea.h                    |   1 +
 OvmfPkg/IntelTdx/IntelTdxX64.dsc              |   8 +
 .../PrePiHobListPointer.c                     |   4 +-
 .../PrePiHobListPointerLibTdx.inf             |   2 +-
 OvmfPkg/Library/PeilessStartupLib/Hob.c       |  26 ++-
 .../PeilessStartupLib/PeilessStartupLib.inf   |   1 +
 OvmfPkg/Library/PlatformInitLib/IntelTdx.c    | 145 +++++++++++-
 OvmfPkg/Library/PlatformInitLib/MemDetect.c   |  27 +++
 .../PlatformInitLib/PlatformInitLib.inf       |   1 +
 OvmfPkg/OvmfPkg.dec                           |   4 +
 OvmfPkg/OvmfPkgX64.dsc                        |   9 +
 OvmfPkg/PlatformPei/MemDetect.c               |   5 +
 OvmfPkg/TdxDxe/TdxDxe.c                       | 103 +++++++++
 OvmfPkg/TdxDxe/TdxDxe.inf                     |   2 +
 .../UefiShellDebug1CommandsLib/MemMap.c       |   6 +
 27 files changed, 666 insertions(+), 19 deletions(-)
 create mode 100644 MdeModulePkg/Include/Pi/PrePiHob.h
 create mode 100644 MdePkg/Include/Protocol/MemoryAccept.h

-- 
2.29.2.windows.2



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#90216): https://edk2.groups.io/g/devel/message/90216
Mute This Topic: https://groups.io/mt/91570192/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH 00/14] Introduce Lazy-accept for Tdx guest
Posted by Min Xu 2 years, 3 months ago
Hi, All
Since the lazy-accept patch-set was sent for review, there are a lot of discussions in multiple OpenSource-Communities (EDK2/QEMU/LinuxKernel). 

Below are the summaries of the discussions/proposals
1. Accept memory under 4G
  - Tom Lendacky's suggestion for SEV-SNP is to pre-accept all memory under 4GB to make all complexity go way. 
     https://edk2.groups.io/g/devel/message/90539 
  - Gerd comments that before the automatic negotiation is available, we use some config option and then accept all memory below 4G or accept all memory.
     https://www.mail-archive.com/qemu-devel@nongnu.org/msg894393.html

2. GetMemoryMapEx / OsIndications
  - https://bugzilla.tianocore.org/show_bug.cgi?id=3987
  - GetMemoryMapEx: cannot resolve the boot-loader because it doesn't know about the kernel's capability.
  - OsIndications & self-report kernel header bit could be a full solution.
 
3.  fw_cfg
  - Add new fw_cfg item (opt/ovmf/AcceptAllMemory) to indicate how to handle the unaccepted memory.
      > True - accept all the memory
      > False - don't accept the memory
      > Default - It allows the firmware to choose depending on various factors.
  - Glaze has submit the patch https://lore.kernel.org/all/20220620223300.1555849-1-dionnaglaze@google.com/
 
4.Put some information in kernel image.  (Peter Gonda)
  - Put information into UTS_VERSION which can be read from the buld bzImage. https://www.spinics.net/lists/linux-mm/msg302506.html
  - Glaze denied it. https://www.spinics.net/lists/linux-mm/msg304973.html
 
5. A protocol kernel call from EFI stub before EBS().(Ard)
  - It doesn't work if boot loader is in the boot flow. 
  - https://www.spinics.net/lists/linux-mm/msg304896.html

Among the above 5 proposals, Proposal 1) is different from the other 4 proposals.
Proposal 1) uses a config option and hence there are 2 kinds of OVMF: OVMF-with-LazyAccept and OVMF-with-FullMemory. End user should be aware of it and choose the correct OVMF for the linux kernel (LazyAccept enlightened or not).
Proposal 3) only works for QEMU because of fw_cfg.

So essentially there are 2 questions about the lazy-accept.
1. If Lazy-accept capability should be automatically negotiated?
2. How to implement the Lazy-Accept in OVMF? 

I wonder if lazy-accept feature can be split into 2 stages. 
1. In first stage there is a config option to indicate if lazy-accept is enabled or not.
2. In the second stage the automatic negotiation is introduced so that lazy-accept is enabled or not by the negotiation result.

Thanks
Min

> -----Original Message-----
> From: Min Xu <min.m.xu@intel.com>
> Sent: Monday, June 6, 2022 11:00 AM
> To: devel@edk2.groups.io
> Cc: Xu, Min M <min.m.xu@intel.com>; Gao, Zhichao
> <zhichao.gao@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>;
> Liu, Zhiguang <zhiguang.liu@intel.com>; Wang, Jian J
> <jian.j.wang@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Ni, Ray
> <ray.ni@intel.com>; Aktas, Erdem <erdemaktas@google.com>; Gerd
> Hoffmann <kraxel@redhat.com>; James Bottomley <jejb@linux.ibm.com>;
> Yao, Jiewen <jiewen.yao@intel.com>; Tom Lendacky
> <thomas.lendacky@amd.com>; Gao, Jiaqi <jiaqi.gao@intel.com>
> Subject: [PATCH 00/14] Introduce Lazy-accept for Tdx guest
> 
> RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3937
> 
> UnacceptedMemory is one of the four defined types of TD memory in Intel
> TDX guest. TDVF must invoke TDCALL [TDG.MEM.PAGE.ACCEPT] the
> unaccepted memory before use it. See [TDVF] Section 7.1.
> TDVF: https://www.intel.com/content/dam/develop/external/us/en/
> documents/tdx-virtual-firmware-design-guide-rev-1.01.pdf
> 
> It is a time-consuming task which impacts the boot performance badly.
> One of the mitigation is the lazy-accept mechanism. That the whole system
> memory is divided into 2 parts, one is accepted in bios phase, the other is
> tagged as EfiGcdMemoryTypeUnaccepted and OS will handle these
> "unaccepted" memories.
> See "UEFI Spec v2.9 Table 7-5 Memory Type Usage before
> ExitBootServices()"
> 
> Patch 1-4:
>   Introduce lazy-accept related definitions.
> 
> Patch 5-6:
>   Update Dxe and shell for unaccepted memory.
> 
> Patch 7 - 11:
>   Update OvmfPkg for unaccepted memory.
> 
> Patch 12 - 13:
>   Introduce EfiMemoryAcceptProtocol and realize it in TdxDxe.
> 
> Patch 14:
>   Update Pool and Page functions to accept memory when OOM occurs.
> 
> Code: https://github.com/mxu9/edk2/tree/lazyaccept.v1
> 
> Cc: Zhichao Gao <zhichao.gao@intel.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Zhiguang Liu <zhiguang.liu@intel.com>
> Cc: Jian J Wang <jian.j.wang@intel.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Ray Ni <ray.ni@intel.com>
> Cc: Erdem Aktas <erdemaktas@google.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: James Bottomley <jejb@linux.ibm.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Jiaqi Gao <jiaqi.gao@intel.com>
> Signed-off-by: Min Xu <min.m.xu@intel.com>
> 
> Jiaqi Gao (2):
>   MdePkg: The prototype definition of EfiMemoryAcceptProtocol
>   MdeModulePkg: Pool and page functions accept memory when OOM
> occurs
> 
> Min M Xu (12):
>   MdeModulePkg: Add PrePiHob.h
>   MdePkg: Increase EFI_RESOURCE_MAX_MEMORY_TYPE
>   OvmfPkg: Use EFI_RESOURCE_MEMORY_UNACCEPTED which defined in
>     MdeModulePkg
>   MdePkg: Add UEFI Unaccepted memory definition
>   MdeModulePkg: Update Dxe to handle unaccepted memory type
>   ShellPkg: Update shell command memmap to show unaccepted memory
>   OvmfPkg: Add PCD and DEFINEs for Lazy Accept page.
>   OvmfPkg: Use PcdOvmfWorkAreaBase instead of PcdSevEsWorkAreaBase
>   OvmfPkg: Add MaxAcceptedMemoryAddress in TDX work area
>   OvmfPkg: Introduce lazy accept in PlatformInitLib and PlatformPei
>   OvmfPkg: Update ConstructFwHobList for lazy accept
>   OvmfPkg: Realize EfiMemoryAcceptProtocol in TdxDxe
> 
>  MdeModulePkg/Core/Dxe/DxeMain.inf             |   1 +
>  MdeModulePkg/Core/Dxe/Gcd/Gcd.c               |   5 +
>  MdeModulePkg/Core/Dxe/Mem/Imem.h              |  16 ++
>  MdeModulePkg/Core/Dxe/Mem/Page.c              | 218 ++++++++++++++++++
>  MdeModulePkg/Core/Dxe/Mem/Pool.c              |  14 ++
>  MdeModulePkg/Include/Pi/PrePiHob.h            |  20 ++
>  MdePkg/Include/Pi/PiDxeCis.h                  |   5 +
>  MdePkg/Include/Pi/PiHob.h                     |  11 +-
>  MdePkg/Include/Protocol/MemoryAccept.h        |  37 +++
>  MdePkg/Include/Uefi/UefiMultiPhase.h          |   5 +
>  MdePkg/MdePkg.dec                             |   3 +
>  OvmfPkg/Include/Library/PlatformInitLib.h     |   6 +
>  OvmfPkg/Include/WorkArea.h                    |   1 +
>  OvmfPkg/IntelTdx/IntelTdxX64.dsc              |   8 +
>  .../PrePiHobListPointer.c                     |   4 +-
>  .../PrePiHobListPointerLibTdx.inf             |   2 +-
>  OvmfPkg/Library/PeilessStartupLib/Hob.c       |  26 ++-
>  .../PeilessStartupLib/PeilessStartupLib.inf   |   1 +
>  OvmfPkg/Library/PlatformInitLib/IntelTdx.c    | 145 +++++++++++-
>  OvmfPkg/Library/PlatformInitLib/MemDetect.c   |  27 +++
>  .../PlatformInitLib/PlatformInitLib.inf       |   1 +
>  OvmfPkg/OvmfPkg.dec                           |   4 +
>  OvmfPkg/OvmfPkgX64.dsc                        |   9 +
>  OvmfPkg/PlatformPei/MemDetect.c               |   5 +
>  OvmfPkg/TdxDxe/TdxDxe.c                       | 103 +++++++++
>  OvmfPkg/TdxDxe/TdxDxe.inf                     |   2 +
>  .../UefiShellDebug1CommandsLib/MemMap.c       |   6 +
>  27 files changed, 666 insertions(+), 19 deletions(-)  create mode 100644
> MdeModulePkg/Include/Pi/PrePiHob.h
>  create mode 100644 MdePkg/Include/Protocol/MemoryAccept.h
> 
> --
> 2.29.2.windows.2



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#92332): https://edk2.groups.io/g/devel/message/92332
Mute This Topic: https://groups.io/mt/91570192/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH 00/14] Introduce Lazy-accept for Tdx guest
Posted by Gerd Hoffmann 2 years, 3 months ago
  Hi,

> 3.  fw_cfg
>   - Add new fw_cfg item (opt/ovmf/AcceptAllMemory) to indicate how to handle the unaccepted memory.
>       > True - accept all the memory
>       > False - don't accept the memory
>       > Default - It allows the firmware to choose depending on various factors.
>   - Glaze has submit the patch https://lore.kernel.org/all/20220620223300.1555849-1-dionnaglaze@google.com/

> Proposal 3) only works for QEMU because of fw_cfg.

Well, while that is true for the patch at hand it doesn't have to be
that way.  We can also simply store the config option in a EFI variable.
Wire up a HII configuration so it can be changed via firmware setup.
Allow setting the EFI variable from fw_cfg, so qemu users can set that
on the qemu command line too (and possibly have similar mechanisms for
other hypervisors, hello cloudhv).

> I wonder if lazy-accept feature can be split into 2 stages. 
> 1. In first stage there is a config option to indicate if lazy-accept is enabled or not.
> 2. In the second stage the automatic negotiation is introduced so that lazy-accept is enabled or not by the negotiation result.

Absolutely.  That is one of the reasons why I suggested to have a
true/false/default config option instead of just true/false.

When the first stage is implemented "default" behavior would be fixed
(either hard-coded or a compile-time option).

When the second stage is implemented "default" behavior would be
dynamic, depending on the negotiation result.

take care,
  Gerd



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#92469): https://edk2.groups.io/g/devel/message/92469
Mute This Topic: https://groups.io/mt/91570192/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH 00/14] Introduce Lazy-accept for Tdx guest
Posted by Dionna Glaze via groups.io 2 years, 1 month ago
I have a working stack with proposal 5. I was using a version of grub that
didn't use Linux's EFI handover protocol, and I wasn't signalling the
unaccepted memory behavior at the right place (before EBS).

Many thanks to everyone who contributed to this discussion and brought some
creative ideas to the table. Thanks especially to Ard, for multiple
consultations.
With proposal 5, we won't need any control plane annotation for disabling
unaccepted memory, and users will always get safe behavior.

I'll have patches out to OVMF (based on this patch series) and to Linux
shortly.

On Mon, Aug 15, 2022 at 11:47 PM Gerd Hoffmann <kraxel@redhat.com> wrote:

>   Hi,
>
> > 3.  fw_cfg
> >   - Add new fw_cfg item (opt/ovmf/AcceptAllMemory) to indicate how to
> handle the unaccepted memory.
> >       > True - accept all the memory
> >       > False - don't accept the memory
> >       > Default - It allows the firmware to choose depending on various
> factors.
> >   - Glaze has submit the patch
> https://lore.kernel.org/all/20220620223300.1555849-1-dionnaglaze@google.com/
>
> > Proposal 3) only works for QEMU because of fw_cfg.
>
> Well, while that is true for the patch at hand it doesn't have to be
> that way.  We can also simply store the config option in a EFI variable.
> Wire up a HII configuration so it can be changed via firmware setup.
> Allow setting the EFI variable from fw_cfg, so qemu users can set that
> on the qemu command line too (and possibly have similar mechanisms for
> other hypervisors, hello cloudhv).
>
> > I wonder if lazy-accept feature can be split into 2 stages.
> > 1. In first stage there is a config option to indicate if lazy-accept is
> enabled or not.
> > 2. In the second stage the automatic negotiation is introduced so that
> lazy-accept is enabled or not by the negotiation result.
>
> Absolutely.  That is one of the reasons why I suggested to have a
> true/false/default config option instead of just true/false.
>
> When the first stage is implemented "default" behavior would be fixed
> (either hard-coded or a compile-time option).
>
> When the second stage is implemented "default" behavior would be
> dynamic, depending on the negotiation result.
>
> take care,
>   Gerd
>
>

-- 
-Dionna Glaze, PhD (she/her)


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#94128): https://edk2.groups.io/g/devel/message/94128
Mute This Topic: https://groups.io/mt/91570192/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-