ArmPkg/Library/ArmLib/AArch64/AArch64Support.S | 4 +++- ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S | 2 ++ 2 files changed, 5 insertions(+), 1 deletion(-)
Even with MMU turned off, instruction cache can speculate
and fetch instructions. This can cause a crash if region
being executed has been modified recently. With this patch,
we ensure that instruction cache is invalidated right after
MMU has been enabled and any potentially stale instruction
fetched earlier has been discarded.
This is specially helpful when the memory attributes of a
region in MMU are being changed and some instructions
operating on the region are prefetched in the instruction
cache.
Signed-off-by: Ashish Singhal <ashishsingha@nvidia.com>
---
ArmPkg/Library/ArmLib/AArch64/AArch64Support.S | 4 +++-
ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S | 2 ++
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/ArmPkg/Library/ArmLib/AArch64/AArch64Support.S b/ArmPkg/Library/ArmLib/AArch64/AArch64Support.S
index d3cc1e8671..047192ec91 100644
--- a/ArmPkg/Library/ArmLib/AArch64/AArch64Support.S
+++ b/ArmPkg/Library/ArmLib/AArch64/AArch64Support.S
@@ -89,7 +89,9 @@ ASM_FUNC(ArmEnableMmu)
dsb nsh
isb
msr sctlr_el3, x0 // Write back
-4: isb
+4: ic iallu
+ dsb nsh
+ isb
ret
diff --git a/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S b/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S
index 66ebca571e..4fe75ec841 100644
--- a/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S
+++ b/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S
@@ -37,6 +37,8 @@
// re-enable the MMU
msr sctlr_el\el, x8
+ ic iallu
+ dsb nsh
isb
.endm
--
2.17.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#87030): https://edk2.groups.io/g/devel/message/87030
Mute This Topic: https://groups.io/mt/89405016/1787277
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org]
-=-=-=-=-=-=-=-=-=-=-=-
On Sat, 26 Feb 2022 04:43:37 +0000, Ashish Singhal <ashishsingha@nvidia.com> wrote: > > Even with MMU turned off, instruction cache can speculate > and fetch instructions. This can cause a crash if region > being executed has been modified recently. With this patch, Modified by what? > we ensure that instruction cache is invalidated right after > MMU has been enabled and any potentially stale instruction > fetched earlier has been discarded. > > This is specially helpful when the memory attributes of a > region in MMU are being changed and some instructions Changed from what to what else? Are you concerned with the content of the memory being changed? Or by the attribute being changed? Or both? > operating on the region are prefetched in the instruction > cache. I don't see how this fixes anything. Yes, speculation occurs. But if your icache contains crap, how is it safe to first enable the MMU first and then nuke the icache? You could well be executing garbage at that point. Worse case, and assuming that you have an aliasing VIVT icache, this will invalidate fetches that would alias with the layout of the memory once the MMU is on. But as far as I know, EDK2 is entirely identity mapped. I also don't think it uses instruction patching. Finally, if you see speculative accesses on regions that shouldn't be accessed as such, it could well be because the code is placed too close to such a region, as mentioned in the ARM ARM (DDI0487H_a, page D5-4828): <quote> Behavior of instruction fetches when all associated stages of translation are disabled [...] To ensure architectural compliance, software must ensure that both of the following apply: • Instructions that will be executed when all associated stages of address translation are disabled are located in blocks of the address space, of the translation granule size, that contain only memory that is tolerant to speculative accesses. • Each block of the address space, of the translation granule size, that immediately follows a similar block that holds instructions that will be executed when all associated stages address translation are disabled, contains only memory that is tolerant to speculative accesses. </quote> Thanks, M -- Without deviation from the norm, progress is not possible. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#87103): https://edk2.groups.io/g/devel/message/87103 Mute This Topic: https://groups.io/mt/89405016/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
@Marc Zyngier<mailto:maz@kernel.org> Thanks for your response. I have provided some comments inline. Please note that I am still not saying what I am doing is a solution and I could still just be masking the problem at hand. Please provide me with some suggestions that I can try for the problem being discussed on the email chain for the first version of this patch where you are already there. Thanks Ashish ________________________________ From: Marc Zyngier <maz@kernel.org> Sent: Saturday, February 26, 2022 2:18 PM To: Ashish Singhal <ashishsingha@nvidia.com> Cc: devel@edk2.groups.io <devel@edk2.groups.io>; quic_llindhol@quicinc.com <quic_llindhol@quicinc.com>; ardb+tianocore@kernel.org <ardb+tianocore@kernel.org>; sami.mujawar@arm.com <sami.mujawar@arm.com> Subject: Re: [PATCH v2] ArmPkg: Invalidate Instruction Cache On MMU Enable External email: Use caution opening links or attachments On Sat, 26 Feb 2022 04:43:37 +0000, Ashish Singhal <ashishsingha@nvidia.com> wrote: > > Even with MMU turned off, instruction cache can speculate > and fetch instructions. This can cause a crash if region > being executed has been modified recently. With this patch, Modified by what? [Singhal, Ashish]: Modified by MMU code in terms of their memory attributes. > we ensure that instruction cache is invalidated right after > MMU has been enabled and any potentially stale instruction > fetched earlier has been discarded. > > This is specially helpful when the memory attributes of a > region in MMU are being changed and some instructions Changed from what to what else? Are you concerned with the content of the memory being changed? Or by the attribute being changed? Or both? [Singhal, Ashish]: I am concerned with the attributes being changed. UEFI drivers are dispatched in RWE memory and then memory attributes are changed to ROE for the code section and RWnE for the data section. > operating on the region are prefetched in the instruction > cache. I don't see how this fixes anything. Yes, speculation occurs. But if your icache contains crap, how is it safe to first enable the MMU first and then nuke the icache? You could well be executing garbage at that point. Worse case, and assuming that you have an aliasing VIVT icache, this will invalidate fetches that would alias with the layout of the memory once the MMU is on. But as far as I know, EDK2 is entirely identity mapped. I also don't think it uses instruction patching. Finally, if you see speculative accesses on regions that shouldn't be accessed as such, it could well be because the code is placed too close to such a region, as mentioned in the ARM ARM (DDI0487H_a, page D5-4828): <quote> Behavior of instruction fetches when all associated stages of translation are disabled [...] To ensure architectural compliance, software must ensure that both of the following apply: • Instructions that will be executed when all associated stages of address translation are disabled are located in blocks of the address space, of the translation granule size, that contain only memory that is tolerant to speculative accesses. • Each block of the address space, of the translation granule size, that immediately follows a similar block that holds instructions that will be executed when all associated stages address translation are disabled, contains only memory that is tolerant to speculative accesses. </quote> [Singhal, Ashish]: I do not think this is an issue here. Thanks, M -- Without deviation from the norm, progress is not possible. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#87034): https://edk2.groups.io/g/devel/message/87034 Mute This Topic: https://groups.io/mt/89405016/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
@Marc Zyngier<mailto:maz@kernel.org> @ardb+tianocore@kernel.org<mailto:ardb+tianocore@kernel.org> I am going to look at things a bit more deeply and try to find out the exact scenario when the issue is occurring and seek help based on that. Meanwhile, you can ignore both the patches I have sent for this issue till now. Thanks Ashish ________________________________ From: Ashish Singhal <ashishsingha@nvidia.com> Sent: Saturday, February 26, 2022 2:48 PM To: Marc Zyngier <maz@kernel.org> Cc: devel@edk2.groups.io <devel@edk2.groups.io>; quic_llindhol@quicinc.com <quic_llindhol@quicinc.com>; ardb+tianocore@kernel.org <ardb+tianocore@kernel.org>; sami.mujawar@arm.com <sami.mujawar@arm.com> Subject: Re: [PATCH v2] ArmPkg: Invalidate Instruction Cache On MMU Enable @Marc Zyngier<mailto:maz@kernel.org> Thanks for your response. I have provided some comments inline. Please note that I am still not saying what I am doing is a solution and I could still just be masking the problem at hand. Please provide me with some suggestions that I can try for the problem being discussed on the email chain for the first version of this patch where you are already there. Thanks Ashish ________________________________ From: Marc Zyngier <maz@kernel.org> Sent: Saturday, February 26, 2022 2:18 PM To: Ashish Singhal <ashishsingha@nvidia.com> Cc: devel@edk2.groups.io <devel@edk2.groups.io>; quic_llindhol@quicinc.com <quic_llindhol@quicinc.com>; ardb+tianocore@kernel.org <ardb+tianocore@kernel.org>; sami.mujawar@arm.com <sami.mujawar@arm.com> Subject: Re: [PATCH v2] ArmPkg: Invalidate Instruction Cache On MMU Enable External email: Use caution opening links or attachments On Sat, 26 Feb 2022 04:43:37 +0000, Ashish Singhal <ashishsingha@nvidia.com> wrote: > > Even with MMU turned off, instruction cache can speculate > and fetch instructions. This can cause a crash if region > being executed has been modified recently. With this patch, Modified by what? [Singhal, Ashish]: Modified by MMU code in terms of their memory attributes. > we ensure that instruction cache is invalidated right after > MMU has been enabled and any potentially stale instruction > fetched earlier has been discarded. > > This is specially helpful when the memory attributes of a > region in MMU are being changed and some instructions Changed from what to what else? Are you concerned with the content of the memory being changed? Or by the attribute being changed? Or both? [Singhal, Ashish]: I am concerned with the attributes being changed. UEFI drivers are dispatched in RWE memory and then memory attributes are changed to ROE for the code section and RWnE for the data section. > operating on the region are prefetched in the instruction > cache. I don't see how this fixes anything. Yes, speculation occurs. But if your icache contains crap, how is it safe to first enable the MMU first and then nuke the icache? You could well be executing garbage at that point. Worse case, and assuming that you have an aliasing VIVT icache, this will invalidate fetches that would alias with the layout of the memory once the MMU is on. But as far as I know, EDK2 is entirely identity mapped. I also don't think it uses instruction patching. Finally, if you see speculative accesses on regions that shouldn't be accessed as such, it could well be because the code is placed too close to such a region, as mentioned in the ARM ARM (DDI0487H_a, page D5-4828): <quote> Behavior of instruction fetches when all associated stages of translation are disabled [...] To ensure architectural compliance, software must ensure that both of the following apply: • Instructions that will be executed when all associated stages of address translation are disabled are located in blocks of the address space, of the translation granule size, that contain only memory that is tolerant to speculative accesses. • Each block of the address space, of the translation granule size, that immediately follows a similar block that holds instructions that will be executed when all associated stages address translation are disabled, contains only memory that is tolerant to speculative accesses. </quote> [Singhal, Ashish]: I do not think this is an issue here. Thanks, M -- Without deviation from the norm, progress is not possible. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#87038): https://edk2.groups.io/g/devel/message/87038 Mute This Topic: https://groups.io/mt/89405016/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=-=-=-=-=-=-=-=-=-=-=-
© 2016 - 2024 Red Hat, Inc.