Now that we have all the pieces in place, switch the AArch64 version of
ArmVirtQemu to a mode where the first thing it does out of reset is
enable a preliminary ID map that covers the NOR flash and sufficient
DRAM to create the UEFI page tables as usual.
The advantage of this is that no manipulation of memory occurs any
longer before the MMU is enabled, which removes the need for explicit
coherency management, which is cumbersome and bad for performance.
It also means we no longer need to build all components that may execute
with the MMU off (including BASE libraries) with strict alignment.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
ArmVirtPkg/ArmVirtQemu.dsc | 17 ++++++++++++++---
ArmVirtPkg/ArmVirtQemu.fdf | 2 +-
2 files changed, 15 insertions(+), 4 deletions(-)
diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc
index 302c0d2a4e29..21a321e35794 100644
--- a/ArmVirtPkg/ArmVirtQemu.dsc
+++ b/ArmVirtPkg/ArmVirtQemu.dsc
@@ -63,8 +63,6 @@ [LibraryClasses.common]
QemuFwCfgSimpleParserLib|OvmfPkg/Library/QemuFwCfgSimpleParserLib/QemuFwCfgSimpleParserLib.inf
QemuLoadImageLib|OvmfPkg/Library/GenericQemuLoadImageLib/GenericQemuLoadImageLib.inf
- ArmPlatformLib|ArmPlatformPkg/Library/ArmPlatformLibNull/ArmPlatformLibNull.inf
-
TimerLib|ArmPkg/Library/ArmArchTimerLib/ArmArchTimerLib.inf
NorFlashPlatformLib|ArmVirtPkg/Library/NorFlashQemuLib/NorFlashQemuLib.inf
@@ -92,6 +90,12 @@ [LibraryClasses.common]
TpmPlatformHierarchyLib|SecurityPkg/Library/PeiDxeTpmPlatformHierarchyLibNull/PeiDxeTpmPlatformHierarchyLib.inf
!endif
+[LibraryClasses.AARCH64]
+ ArmPlatformLib|ArmVirtPkg/Library/ArmPlatformLibQemu/ArmPlatformLibQemu.inf
+
+[LibraryClasses.ARM]
+ ArmPlatformLib|ArmPlatformPkg/Library/ArmPlatformLibNull/ArmPlatformLibNull.inf
+
[LibraryClasses.common.PEIM]
ArmVirtMemInfoLib|ArmVirtPkg/Library/QemuVirtMemInfoLib/QemuVirtMemInfoPeiLib.inf
@@ -112,6 +116,8 @@ [LibraryClasses.common.UEFI_DRIVER]
UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf
[BuildOptions]
+ GCC:*_*_AARCH64_CC_XIPFLAGS = -mno-strict-align
+
!include NetworkPkg/NetworkBuildOptions.dsc.inc
################################################################################
@@ -310,7 +316,12 @@ [Components.common]
PcdLib|MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf
}
ArmPlatformPkg/PlatformPei/PlatformPeim.inf
- ArmPlatformPkg/MemoryInitPei/MemoryInitPeim.inf
+ ArmVirtPkg/MemoryInitPei/MemoryInitPeim.inf {
+ <LibraryClasses>
+!if $(ARCH) == AARCH64
+ ArmMmuLib|ArmPkg/Library/ArmMmuLib/ArmMmuPeiLib.inf
+!endif
+ }
ArmPkg/Drivers/CpuPei/CpuPei.inf
MdeModulePkg/Universal/Variable/Pei/VariablePei.inf
diff --git a/ArmVirtPkg/ArmVirtQemu.fdf b/ArmVirtPkg/ArmVirtQemu.fdf
index b5e2253295fe..7f17aeb3ad0d 100644
--- a/ArmVirtPkg/ArmVirtQemu.fdf
+++ b/ArmVirtPkg/ArmVirtQemu.fdf
@@ -107,7 +107,7 @@ [FV.FVMAIN_COMPACT]
INF ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf
INF MdeModulePkg/Core/Pei/PeiMain.inf
INF ArmPlatformPkg/PlatformPei/PlatformPeim.inf
- INF ArmPlatformPkg/MemoryInitPei/MemoryInitPeim.inf
+ INF ArmVirtPkg/MemoryInitPei/MemoryInitPeim.inf
INF ArmPkg/Drivers/CpuPei/CpuPei.inf
INF MdeModulePkg/Universal/PCD/Pei/Pcd.inf
INF MdeModulePkg/Universal/Variable/Pei/VariablePei.inf
--
2.35.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#94335): https://edk2.groups.io/g/devel/message/94335
Mute This Topic: https://groups.io/mt/93922702/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-On Mon, Sep 26, 2022 at 10:25:07AM +0200, Ard Biesheuvel wrote:
> Now that we have all the pieces in place, switch the AArch64 version of
> ArmVirtQemu to a mode where the first thing it does out of reset is
> enable a preliminary ID map that covers the NOR flash and sufficient
> DRAM to create the UEFI page tables as usual.
>
> The advantage of this is that no manipulation of memory occurs any
> longer before the MMU is enabled, which removes the need for explicit
> coherency management, which is cumbersome and bad for performance.
>
> It also means we no longer need to build all components that may execute
> with the MMU off (including BASE libraries) with strict alignment.
After this switch, I'm seeing a Synchronous Exception when launching a
VM, though only on old Cavium ThunderX (CN88XX) systems. I used print
debugging to narrow it down to ArmSetTTBR0(). Initially I thought it
might be related to Cavium Erratum 27456, but that doesn't seem to
make sense because the instruction cache isn't enabled until
later. I tried implementing the same workaround as Linux does anyway
(flush caches after the setting ttbr0) without any luck.
Any idea what is going on there?
-dann
>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> ---
> ArmVirtPkg/ArmVirtQemu.dsc | 17 ++++++++++++++---
> ArmVirtPkg/ArmVirtQemu.fdf | 2 +-
> 2 files changed, 15 insertions(+), 4 deletions(-)
>
> diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc
> index 302c0d2a4e29..21a321e35794 100644
> --- a/ArmVirtPkg/ArmVirtQemu.dsc
> +++ b/ArmVirtPkg/ArmVirtQemu.dsc
> @@ -63,8 +63,6 @@ [LibraryClasses.common]
> QemuFwCfgSimpleParserLib|OvmfPkg/Library/QemuFwCfgSimpleParserLib/QemuFwCfgSimpleParserLib.inf
> QemuLoadImageLib|OvmfPkg/Library/GenericQemuLoadImageLib/GenericQemuLoadImageLib.inf
>
> - ArmPlatformLib|ArmPlatformPkg/Library/ArmPlatformLibNull/ArmPlatformLibNull.inf
> -
> TimerLib|ArmPkg/Library/ArmArchTimerLib/ArmArchTimerLib.inf
> NorFlashPlatformLib|ArmVirtPkg/Library/NorFlashQemuLib/NorFlashQemuLib.inf
>
> @@ -92,6 +90,12 @@ [LibraryClasses.common]
> TpmPlatformHierarchyLib|SecurityPkg/Library/PeiDxeTpmPlatformHierarchyLibNull/PeiDxeTpmPlatformHierarchyLib.inf
> !endif
>
> +[LibraryClasses.AARCH64]
> + ArmPlatformLib|ArmVirtPkg/Library/ArmPlatformLibQemu/ArmPlatformLibQemu.inf
> +
> +[LibraryClasses.ARM]
> + ArmPlatformLib|ArmPlatformPkg/Library/ArmPlatformLibNull/ArmPlatformLibNull.inf
> +
> [LibraryClasses.common.PEIM]
> ArmVirtMemInfoLib|ArmVirtPkg/Library/QemuVirtMemInfoLib/QemuVirtMemInfoPeiLib.inf
>
> @@ -112,6 +116,8 @@ [LibraryClasses.common.UEFI_DRIVER]
> UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf
>
> [BuildOptions]
> + GCC:*_*_AARCH64_CC_XIPFLAGS = -mno-strict-align
> +
> !include NetworkPkg/NetworkBuildOptions.dsc.inc
>
> ################################################################################
> @@ -310,7 +316,12 @@ [Components.common]
> PcdLib|MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf
> }
> ArmPlatformPkg/PlatformPei/PlatformPeim.inf
> - ArmPlatformPkg/MemoryInitPei/MemoryInitPeim.inf
> + ArmVirtPkg/MemoryInitPei/MemoryInitPeim.inf {
> + <LibraryClasses>
> +!if $(ARCH) == AARCH64
> + ArmMmuLib|ArmPkg/Library/ArmMmuLib/ArmMmuPeiLib.inf
> +!endif
> + }
> ArmPkg/Drivers/CpuPei/CpuPei.inf
>
> MdeModulePkg/Universal/Variable/Pei/VariablePei.inf
> diff --git a/ArmVirtPkg/ArmVirtQemu.fdf b/ArmVirtPkg/ArmVirtQemu.fdf
> index b5e2253295fe..7f17aeb3ad0d 100644
> --- a/ArmVirtPkg/ArmVirtQemu.fdf
> +++ b/ArmVirtPkg/ArmVirtQemu.fdf
> @@ -107,7 +107,7 @@ [FV.FVMAIN_COMPACT]
> INF ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf
> INF MdeModulePkg/Core/Pei/PeiMain.inf
> INF ArmPlatformPkg/PlatformPei/PlatformPeim.inf
> - INF ArmPlatformPkg/MemoryInitPei/MemoryInitPeim.inf
> + INF ArmVirtPkg/MemoryInitPei/MemoryInitPeim.inf
> INF ArmPkg/Drivers/CpuPei/CpuPei.inf
> INF MdeModulePkg/Universal/PCD/Pei/Pcd.inf
> INF MdeModulePkg/Universal/Variable/Pei/VariablePei.inf
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#97815): https://edk2.groups.io/g/devel/message/97815
Mute This Topic: https://groups.io/mt/93922702/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
On Thu, 29 Dec 2022 at 22:10, dann frazier <dann.frazier@canonical.com> wrote: > > On Mon, Sep 26, 2022 at 10:25:07AM +0200, Ard Biesheuvel wrote: > > Now that we have all the pieces in place, switch the AArch64 version of > > ArmVirtQemu to a mode where the first thing it does out of reset is > > enable a preliminary ID map that covers the NOR flash and sufficient > > DRAM to create the UEFI page tables as usual. > > > > The advantage of this is that no manipulation of memory occurs any > > longer before the MMU is enabled, which removes the need for explicit > > coherency management, which is cumbersome and bad for performance. > > > > It also means we no longer need to build all components that may execute > > with the MMU off (including BASE libraries) with strict alignment. > > After this switch, I'm seeing a Synchronous Exception when launching a > VM, though only on old Cavium ThunderX (CN88XX) systems. I used print > debugging to narrow it down to ArmSetTTBR0(). Initially I thought it > might be related to Cavium Erratum 27456, but that doesn't seem to > make sense because the instruction cache isn't enabled until > later. I tried implementing the same workaround as Linux does anyway > (flush caches after the setting ttbr0) without any luck. > > Any idea what is going on there? > I suspect it is in fact the same erratum - the I-cache does get enabled almost immediately after reset, and the use of ASIDs for EL1 mappings looks suspiciously like failure mode that required us to disable KPTI for ThunderX on Linux. It is a bit disappointing to have to add workarounds for obsolete platforms in new code, so if I fix this, it will be gated by a -D build option - is that acceptable to you? -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#97864): https://edk2.groups.io/g/devel/message/97864 Mute This Topic: https://groups.io/mt/93922702/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
On Tue, Jan 03, 2023 at 10:02:27AM +0100, Ard Biesheuvel wrote: > On Thu, 29 Dec 2022 at 22:10, dann frazier <dann.frazier@canonical.com> wrote: > > > > On Mon, Sep 26, 2022 at 10:25:07AM +0200, Ard Biesheuvel wrote: > > > Now that we have all the pieces in place, switch the AArch64 version of > > > ArmVirtQemu to a mode where the first thing it does out of reset is > > > enable a preliminary ID map that covers the NOR flash and sufficient > > > DRAM to create the UEFI page tables as usual. > > > > > > The advantage of this is that no manipulation of memory occurs any > > > longer before the MMU is enabled, which removes the need for explicit > > > coherency management, which is cumbersome and bad for performance. > > > > > > It also means we no longer need to build all components that may execute > > > with the MMU off (including BASE libraries) with strict alignment. > > > > After this switch, I'm seeing a Synchronous Exception when launching a > > VM, though only on old Cavium ThunderX (CN88XX) systems. I used print > > debugging to narrow it down to ArmSetTTBR0(). Initially I thought it > > might be related to Cavium Erratum 27456, but that doesn't seem to > > make sense because the instruction cache isn't enabled until > > later. I tried implementing the same workaround as Linux does anyway > > (flush caches after the setting ttbr0) without any luck. > > > > Any idea what is going on there? > > > > I suspect it is in fact the same erratum - the I-cache does get > enabled almost immediately after reset, and the use of ASIDs for EL1 > mappings looks suspiciously like failure mode that required us to > disable KPTI for ThunderX on Linux. > > It is a bit disappointing to have to add workarounds for obsolete > platforms in new code, so if I fix this, it will be gated by a -D > build option - is that acceptable to you? It is. fwiw, here's what I tried w/o luck: diff --git a/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S b/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S index ba0ec5682b..1a94a9782c 100644 --- a/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S +++ b/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S @@ -56,10 +56,30 @@ ASM_FUNC(ArmReadAuxCr) ASM_FUNC(ArmSetTTBR0) EL1_OR_EL2_OR_EL3(x1) 1:msr ttbr0_el1, x0 // Translation Table Base Reg 0 (TTBR0) + isb + nop + nop + nop + ic iallu + dsb nsh + isb b 4f 2:msr ttbr0_el2, x0 // Translation Table Base Reg 0 (TTBR0) + isb + nop + nop + nop + ic iallu + dsb nsh + isb b 4f 3:msr ttbr0_el3, x0 // Translation Table Base Reg 0 (TTBR0) + isb + nop + nop + nop + ic iallu + dsb nsh 4:isb ret -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#97889): https://edk2.groups.io/g/devel/message/97889 Mute This Topic: https://groups.io/mt/93922702/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
© 2016 - 2026 Red Hat, Inc.