[edk2-devel] [PATCH v1 0/9] Add ImagePropertiesRecordLib and Fix MAT Bugs

Taylor Beebe posted 9 patches 9 months, 2 weeks ago
Failed in applying to current master (apply log)
There is a newer version of this series
MdeModulePkg/Core/Dxe/Misc/MemoryAttributesTable.c                                              | 786 +---------------
MdeModulePkg/Core/Dxe/Misc/MemoryProtection.c                                                   |  24 +-
MdeModulePkg/Core/PiSmmCore/MemoryAttributesTable.c                                             | 785 +---------------
MdeModulePkg/Library/ImagePropertiesRecordLib/ImagePropertiesRecordLib.c                        | 805 +++++++++++++++++
MdeModulePkg/Library/ImagePropertiesRecordLib/UnitTest/ImagePropertiesRecordLibUnitTestHost.c   | 938 ++++++++++++++++++++
UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c                                              |  19 +-
ArmVirtPkg/ArmVirt.dsc.inc                                                                      |   1 +
EmulatorPkg/EmulatorPkg.dsc                                                                     |   1 +
MdeModulePkg/Core/Dxe/DxeMain.h                                                                 |  20 -
MdeModulePkg/Core/Dxe/DxeMain.inf                                                               |   1 +
MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf                                                       |   1 +
MdeModulePkg/Include/Library/ImagePropertiesRecordLib.h                                         | 151 ++++
MdeModulePkg/Library/ImagePropertiesRecordLib/ImagePropertiesRecordLib.inf                      |  28 +
MdeModulePkg/Library/ImagePropertiesRecordLib/UnitTest/ImagePropertiesRecordLibUnitTestHost.inf |  35 +
MdeModulePkg/MdeModulePkg.dec                                                                   |   5 +
MdeModulePkg/MdeModulePkg.dsc                                                                   |   2 +
MdeModulePkg/Test/MdeModulePkgHostTest.dsc                                                      |   5 +
OvmfPkg/AmdSev/AmdSevX64.dsc                                                                    |   1 +
OvmfPkg/Bhyve/BhyveX64.dsc                                                                      |   1 +
OvmfPkg/CloudHv/CloudHvX64.dsc                                                                  |   1 +
OvmfPkg/IntelTdx/IntelTdxX64.dsc                                                                |   1 +
OvmfPkg/Microvm/MicrovmX64.dsc                                                                  |   1 +
OvmfPkg/OvmfPkgIa32.dsc                                                                         |   1 +
OvmfPkg/OvmfPkgIa32X64.dsc                                                                      |   1 +
OvmfPkg/OvmfPkgX64.dsc                                                                          |   1 +
OvmfPkg/OvmfXen.dsc                                                                             |   1 +
OvmfPkg/RiscVVirt/RiscVVirtQemu.dsc                                                             |   1 +
UefiPayloadPkg/UefiPayloadPkg.dsc                                                               |   1 +
28 files changed, 2039 insertions(+), 1579 deletions(-)
create mode 100644 MdeModulePkg/Library/ImagePropertiesRecordLib/ImagePropertiesRecordLib.c
create mode 100644 MdeModulePkg/Library/ImagePropertiesRecordLib/UnitTest/ImagePropertiesRecordLibUnitTestHost.c
create mode 100644 MdeModulePkg/Include/Library/ImagePropertiesRecordLib.h
create mode 100644 MdeModulePkg/Library/ImagePropertiesRecordLib/ImagePropertiesRecordLib.inf
create mode 100644 MdeModulePkg/Library/ImagePropertiesRecordLib/UnitTest/ImagePropertiesRecordLibUnitTestHost.inf
[edk2-devel] [PATCH v1 0/9] Add ImagePropertiesRecordLib and Fix MAT Bugs
Posted by Taylor Beebe 9 months, 2 weeks ago
Reference: https://github.com/tianocore/edk2/pull/4590
Bugzilla: https://bugzilla.tianocore.org/show_bug.cgi?id=4492

The UEFI and SMM MAT logic contains duplicate logic for manipulating image
properties records which is used to track runtime images.
This patch series adds a new library, ImagePropertiesRecordLib,
which consolidates this logic and fixes the bugs which currently exist in
the MAT logic.

The first patch adds the ImagePropertiesRecordLib implementation which
is a copy of the UEFI MAT logic with minor modifications to remove the
reliance on globabl variables and make the code unit testable.

The second patch adds a unit test for the ImagePropertiesRecordLib. The
logic tests various potential layouts of the EFI memory map and runtime
images. 3/4 of these tests will fail which demonstrates the MAT logic
bugs.

The third patch fixes the logic in the ImagePropertiesRecordLib so
that all of the unit tests pass and the MAT logic can be fixed by
using the library.

The remaining patches add library instances to DSC files and remove
the image properties record logic from the SMM and UEFI MAT logic.

Cc: Andrew Fish <afish@apple.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Dandan Bi <dandan.bi@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Guo Dong <guo.dong@intel.com>
Cc: Gua Guo <gua.guo@intel.com>
Cc: James Lu <james.lu@intel.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Sami Mujawar <sami.mujawar@arm.com>
Cc: Sean Rhodes <sean@starlabs.systems>

Taylor Beebe (9):
  MdeModulePkg: Add ImagePropertiesRecordLib
  MdeModulePkg: Add ImagePropertiesRecordLib Host-Based Unit Test
  MdeModulePkg: Fix Bugs in MAT Logic
  ArmVirtPkg: Add ImagePropertiesRecordLib Instance
  EmulatorPkg: Add ImagePropertiesRecordLib Instance
  OvmfPkg: Add ImagePropertiesRecordLib Instance
  UefiPayloadPkg: Add ImagePropertiesRecordLib Instance
  UefiCpuPkg: Use Attribute From SMM MemoryAttributesTable if Nonzero
  MdeModulePkg: Update UEFI and SMM MAT Logic To Use
    ImagePropertiesRecordLib

 MdeModulePkg/Core/Dxe/Misc/MemoryAttributesTable.c                                              | 786 +---------------
 MdeModulePkg/Core/Dxe/Misc/MemoryProtection.c                                                   |  24 +-
 MdeModulePkg/Core/PiSmmCore/MemoryAttributesTable.c                                             | 785 +---------------
 MdeModulePkg/Library/ImagePropertiesRecordLib/ImagePropertiesRecordLib.c                        | 805 +++++++++++++++++
 MdeModulePkg/Library/ImagePropertiesRecordLib/UnitTest/ImagePropertiesRecordLibUnitTestHost.c   | 938 ++++++++++++++++++++
 UefiCpuPkg/PiSmmCpuDxeSmm/SmmCpuMemoryManagement.c                                              |  19 +-
 ArmVirtPkg/ArmVirt.dsc.inc                                                                      |   1 +
 EmulatorPkg/EmulatorPkg.dsc                                                                     |   1 +
 MdeModulePkg/Core/Dxe/DxeMain.h                                                                 |  20 -
 MdeModulePkg/Core/Dxe/DxeMain.inf                                                               |   1 +
 MdeModulePkg/Core/PiSmmCore/PiSmmCore.inf                                                       |   1 +
 MdeModulePkg/Include/Library/ImagePropertiesRecordLib.h                                         | 151 ++++
 MdeModulePkg/Library/ImagePropertiesRecordLib/ImagePropertiesRecordLib.inf                      |  28 +
 MdeModulePkg/Library/ImagePropertiesRecordLib/UnitTest/ImagePropertiesRecordLibUnitTestHost.inf |  35 +
 MdeModulePkg/MdeModulePkg.dec                                                                   |   5 +
 MdeModulePkg/MdeModulePkg.dsc                                                                   |   2 +
 MdeModulePkg/Test/MdeModulePkgHostTest.dsc                                                      |   5 +
 OvmfPkg/AmdSev/AmdSevX64.dsc                                                                    |   1 +
 OvmfPkg/Bhyve/BhyveX64.dsc                                                                      |   1 +
 OvmfPkg/CloudHv/CloudHvX64.dsc                                                                  |   1 +
 OvmfPkg/IntelTdx/IntelTdxX64.dsc                                                                |   1 +
 OvmfPkg/Microvm/MicrovmX64.dsc                                                                  |   1 +
 OvmfPkg/OvmfPkgIa32.dsc                                                                         |   1 +
 OvmfPkg/OvmfPkgIa32X64.dsc                                                                      |   1 +
 OvmfPkg/OvmfPkgX64.dsc                                                                          |   1 +
 OvmfPkg/OvmfXen.dsc                                                                             |   1 +
 OvmfPkg/RiscVVirt/RiscVVirtQemu.dsc                                                             |   1 +
 UefiPayloadPkg/UefiPayloadPkg.dsc                                                               |   1 +
 28 files changed, 2039 insertions(+), 1579 deletions(-)
 create mode 100644 MdeModulePkg/Library/ImagePropertiesRecordLib/ImagePropertiesRecordLib.c
 create mode 100644 MdeModulePkg/Library/ImagePropertiesRecordLib/UnitTest/ImagePropertiesRecordLibUnitTestHost.c
 create mode 100644 MdeModulePkg/Include/Library/ImagePropertiesRecordLib.h
 create mode 100644 MdeModulePkg/Library/ImagePropertiesRecordLib/ImagePropertiesRecordLib.inf
 create mode 100644 MdeModulePkg/Library/ImagePropertiesRecordLib/UnitTest/ImagePropertiesRecordLibUnitTestHost.inf

-- 
2.41.0.windows.2



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107021): https://edk2.groups.io/g/devel/message/107021
Mute This Topic: https://groups.io/mt/100221556/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] [PATCH v1 0/9] Add ImagePropertiesRecordLib and Fix MAT Bugs
Posted by Ard Biesheuvel 9 months, 2 weeks ago
On Tue, 18 Jul 2023 at 20:40, Taylor Beebe <t@taylorbeebe.com> wrote:
>
> Reference: https://github.com/tianocore/edk2/pull/4590
> Bugzilla: https://bugzilla.tianocore.org/show_bug.cgi?id=4492
>
> The UEFI and SMM MAT logic contains duplicate logic for manipulating image
> properties records which is used to track runtime images.
> This patch series adds a new library, ImagePropertiesRecordLib,
> which consolidates this logic and fixes the bugs which currently exist in
> the MAT logic.
>
> The first patch adds the ImagePropertiesRecordLib implementation which
> is a copy of the UEFI MAT logic with minor modifications to remove the
> reliance on globabl variables and make the code unit testable.
>
> The second patch adds a unit test for the ImagePropertiesRecordLib. The
> logic tests various potential layouts of the EFI memory map and runtime
> images. 3/4 of these tests will fail which demonstrates the MAT logic
> bugs.
>
> The third patch fixes the logic in the ImagePropertiesRecordLib so
> that all of the unit tests pass and the MAT logic can be fixed by
> using the library.
>
> The remaining patches add library instances to DSC files and remove
> the image properties record logic from the SMM and UEFI MAT logic.
>

This all looks fine to me, but this is another series of the pattern
- break out some functionality into a new lib class
- provide a single implementation of that lib class
- track down every DSC in existence and add the same library class
resolution to each.

Could we *please* have a way for library classes to be declared with a
default resolution? That way, series such as this one will be a lot
leaner, and as a bonus, I am sure that there is a lot of boilerplate
that can be removed from existing DSCs for library classes that only
have a single implementation.

For this series,

Acked-by: Ard Biesheuvel <ardb@kernel.org>


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107031): https://edk2.groups.io/g/devel/message/107031
Mute This Topic: https://groups.io/mt/100221556/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
回复: [edk2-devel] [PATCH v1 0/9] Add ImagePropertiesRecordLib and Fix MAT Bugs
Posted by gaoliming via groups.io 9 months, 1 week ago
Ard:
  Now, there is no compatible way to introduce new library class for the basic module. One possible solution is to define the default library instance in package dec file together with the library class declaration. 

Beebe:
  This patch set looks good. But, the last patch 9/9 will require every platform to be updated. I may suggest to merge the first 8 patches for this stable tag 202308. The last one 9/9 can be merged for next stable tag 202311. Then, the platform owner have time to update their DSC first. 

Thanks
Liming
> -----邮件原件-----
> 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Ard
> Biesheuvel
> 发送时间: 2023年7月19日 4:18
> 收件人: Taylor Beebe <t@taylorbeebe.com>
> 抄送: devel@edk2.groups.io; Andrew Fish <afish@apple.com>; Dandan Bi
> <dandan.bi@intel.com>; Eric Dong <eric.dong@intel.com>; Gerd Hoffmann
> <kraxel@redhat.com>; Guo Dong <guo.dong@intel.com>; Gua Guo
> <gua.guo@intel.com>; James Lu <james.lu@intel.com>; Jian J Wang
> <jian.j.wang@intel.com>; Jiewen Yao <jiewen.yao@intel.com>; Jordan Justen
> <jordan.l.justen@intel.com>; Leif Lindholm <quic_llindhol@quicinc.com>;
> Liming Gao <gaoliming@byosoft.com.cn>; Rahul Kumar
> <rahul1.kumar@intel.com>; Ray Ni <ray.ni@intel.com>; Sami Mujawar
> <sami.mujawar@arm.com>; Sean Rhodes <sean@starlabs.systems>
> 主题: Re: [edk2-devel] [PATCH v1 0/9] Add ImagePropertiesRecordLib and Fix
> MAT Bugs
> 
> On Tue, 18 Jul 2023 at 20:40, Taylor Beebe <t@taylorbeebe.com> wrote:
> >
> > Reference: https://github.com/tianocore/edk2/pull/4590
> > Bugzilla: https://bugzilla.tianocore.org/show_bug.cgi?id=4492
> >
> > The UEFI and SMM MAT logic contains duplicate logic for manipulating
> image
> > properties records which is used to track runtime images.
> > This patch series adds a new library, ImagePropertiesRecordLib,
> > which consolidates this logic and fixes the bugs which currently exist in
> > the MAT logic.
> >
> > The first patch adds the ImagePropertiesRecordLib implementation which
> > is a copy of the UEFI MAT logic with minor modifications to remove the
> > reliance on globabl variables and make the code unit testable.
> >
> > The second patch adds a unit test for the ImagePropertiesRecordLib. The
> > logic tests various potential layouts of the EFI memory map and runtime
> > images. 3/4 of these tests will fail which demonstrates the MAT logic
> > bugs.
> >
> > The third patch fixes the logic in the ImagePropertiesRecordLib so
> > that all of the unit tests pass and the MAT logic can be fixed by
> > using the library.
> >
> > The remaining patches add library instances to DSC files and remove
> > the image properties record logic from the SMM and UEFI MAT logic.
> >
> 
> This all looks fine to me, but this is another series of the pattern
> - break out some functionality into a new lib class
> - provide a single implementation of that lib class
> - track down every DSC in existence and add the same library class
> resolution to each.
> 
> Could we *please* have a way for library classes to be declared with a
> default resolution? That way, series such as this one will be a lot
> leaner, and as a bonus, I am sure that there is a lot of boilerplate
> that can be removed from existing DSCs for library classes that only
> have a single implementation.
> 
> For this series,
> 
> Acked-by: Ard Biesheuvel <ardb@kernel.org>
> 
> 
> 
> 





-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107036): https://edk2.groups.io/g/devel/message/107036
Mute This Topic: https://groups.io/mt/100230477/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
Re: 回复: [edk2-devel] [PATCH v1 0/9] Add ImagePropertiesRecordLib and Fix MAT Bugs
Posted by Taylor Beebe 9 months, 1 week ago

On 7/18/23 10:23 PM, gaoliming via groups.io wrote:
> Ard:
>    Now, there is no compatible way to introduce new library class for the basic module. One possible solution is to define the default library instance in package dec file together with the library class declaration.
> 
> Beebe:
>    This patch set looks good. But, the last patch 9/9 will require every platform to be updated. I may suggest to merge the first 8 patches for this stable tag 202308. The last one 9/9 can be merged for next stable tag 202311. Then, the platform owner have time to update their DSC first.

Good idea! I did just notice that a one-line change in Patch 3/9 should
instead be in patch 9/9. I'll send out a v2 now.

> 
> Thanks
> Liming


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107065): https://edk2.groups.io/g/devel/message/107065
Mute This Topic: https://groups.io/mt/100230477/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-