Documentation/admin-guide/kernel-parameters.txt | 9 +++++++++ arch/arm64/kernel/cpufeature.c | 14 +++++++++++++- 2 files changed, 22 insertions(+), 1 deletion(-)
Add kernel parameter to allow system-wide EL0 access to IMPDEF system
regregisters and instructions without trapping to EL1/EL2. Since trap
overhead will compromises benefits, and it's even worse in
virtualization on CPU where certain IMPDEF registers and instructions
are designed for EL0 performance use.
More early discussion could be found from link below.
Link: https://lore.kernel.org/all/24afb8de-622a-4865-bd8e-8e89ccfff8f4@huawei.com/
Tested-by: Xinyu Zheng <zhengxinyu6@huawei.com>
Signed-off-by: Liao Chang <liaochang1@huawei.com>
---
Documentation/admin-guide/kernel-parameters.txt | 9 +++++++++
arch/arm64/kernel/cpufeature.c | 14 +++++++++++++-
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 5a7a83c411e9..11ffa9f7b972 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -326,6 +326,15 @@
See Documentation/arch/arm64/asymmetric-32bit.rst for more
information.
+ no_trap_el0_impdef [Arm64,EARLY]
+ Allow system-wide EL0 access to IMPDEF system registers
+ and instructions without trapping to EL1/EL2.
+ Since trap overhead compromises benefits, and it's even
+ worse in virtualization on CPU where certain IMPDEF
+ registers and instructions are designed for EL0
+ performance use. This assumes the kernel adds the
+ support for Armv8.8 extension FEAT_TIDCP1.
+
amd_iommu= [HW,X86-64]
Pass parameters to the AMD IOMMU driver in the system.
Possible values are:
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index ef269a5a37e1..d12e35d799ee 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -2459,9 +2459,21 @@ static bool is_kvm_protected_mode(const struct arm64_cpu_capabilities *entry, in
}
#endif /* CONFIG_KVM */
+static bool no_trap_el0_impdef;
+
+static int __init parse_no_trap_el0_impdef(char *p)
+{
+ no_trap_el0_impdef = true;
+ return 0;
+}
+early_param("no_trap_el0_impdef", parse_no_trap_el0_impdef);
+
static void cpu_trap_el0_impdef(const struct arm64_cpu_capabilities *__unused)
{
- sysreg_clear_set(sctlr_el1, 0, SCTLR_EL1_TIDCP);
+ if (no_trap_el0_impdef)
+ sysreg_clear_set(sctlr_el1, SCTLR_EL1_TIDCP, 0);
+ else
+ sysreg_clear_set(sctlr_el1, 0, SCTLR_EL1_TIDCP);
}
static void cpu_enable_dit(const struct arm64_cpu_capabilities *__unused)
--
2.34.1On 2025/10/21 19:54, Liao Chang wrote:
> Add kernel parameter to allow system-wide EL0 access to IMPDEF system
> regregisters and instructions without trapping to EL1/EL2. Since trap
> overhead will compromises benefits, and it's even worse in
> virtualization on CPU where certain IMPDEF registers and instructions
> are designed for EL0 performance use.
>
> More early discussion could be found from link below.
>
> Link: https://lore.kernel.org/all/24afb8de-622a-4865-bd8e-8e89ccfff8f4@huawei.com/
> Tested-by: Xinyu Zheng <zhengxinyu6@huawei.com>
> Signed-off-by: Liao Chang <liaochang1@huawei.com>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 9 +++++++++
> arch/arm64/kernel/cpufeature.c | 14 +++++++++++++-
> 2 files changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 5a7a83c411e9..11ffa9f7b972 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -326,6 +326,15 @@
> See Documentation/arch/arm64/asymmetric-32bit.rst for more
> information.
>
> + no_trap_el0_impdef [Arm64,EARLY]
> + Allow system-wide EL0 access to IMPDEF system registers
> + and instructions without trapping to EL1/EL2.
> + Since trap overhead compromises benefits, and it's even
> + worse in virtualization on CPU where certain IMPDEF
> + registers and instructions are designed for EL0
> + performance use. This assumes the kernel adds the
> + support for Armv8.8 extension FEAT_TIDCP1.
> +
> amd_iommu= [HW,X86-64]
> Pass parameters to the AMD IOMMU driver in the system.
> Possible values are:
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index ef269a5a37e1..d12e35d799ee 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -2459,9 +2459,21 @@ static bool is_kvm_protected_mode(const struct arm64_cpu_capabilities *entry, in
> }
> #endif /* CONFIG_KVM */
>
> +static bool no_trap_el0_impdef;
> +
> +static int __init parse_no_trap_el0_impdef(char *p)
> +{
> + no_trap_el0_impdef = true;
> + return 0;
> +}
> +early_param("no_trap_el0_impdef", parse_no_trap_el0_impdef);
> +
> static void cpu_trap_el0_impdef(const struct arm64_cpu_capabilities *__unused)
> {
> - sysreg_clear_set(sctlr_el1, 0, SCTLR_EL1_TIDCP);
> + if (no_trap_el0_impdef)
> + sysreg_clear_set(sctlr_el1, SCTLR_EL1_TIDCP, 0);
> + else
> + sysreg_clear_set(sctlr_el1, 0, SCTLR_EL1_TIDCP);
An id override for TIDCP1 seems simpler rather than an early param, and consistent to how we disable other
certain features through boot option like arm64.nosve or arm64.nompam. I see Catalin also suggested in that
way in [1],any reason we cannot do it in that way?
[1] https://lore.kernel.org/all/aHqamaqueuk18NyS@arm.com/
Thanks.
> }
>
> static void cpu_enable_dit(const struct arm64_cpu_capabilities *__unused)
Hi, Yicong
在 2025/10/22 13:26, Yicong Yang 写道:
> On 2025/10/21 19:54, Liao Chang wrote:
>> Add kernel parameter to allow system-wide EL0 access to IMPDEF system
>> regregisters and instructions without trapping to EL1/EL2. Since trap
>> overhead will compromises benefits, and it's even worse in
>> virtualization on CPU where certain IMPDEF registers and instructions
>> are designed for EL0 performance use.
>>
>> More early discussion could be found from link below.
>>
>> Link: https://lore.kernel.org/all/24afb8de-622a-4865-bd8e-8e89ccfff8f4@huawei.com/
>> Tested-by: Xinyu Zheng <zhengxinyu6@huawei.com>
>> Signed-off-by: Liao Chang <liaochang1@huawei.com>
>> ---
>> Documentation/admin-guide/kernel-parameters.txt | 9 +++++++++
>> arch/arm64/kernel/cpufeature.c | 14 +++++++++++++-
>> 2 files changed, 22 insertions(+), 1 deletion(-)
>>
>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>> index 5a7a83c411e9..11ffa9f7b972 100644
>> --- a/Documentation/admin-guide/kernel-parameters.txt
>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>> @@ -326,6 +326,15 @@
>> See Documentation/arch/arm64/asymmetric-32bit.rst for more
>> information.
>>
>> + no_trap_el0_impdef [Arm64,EARLY]
>> + Allow system-wide EL0 access to IMPDEF system registers
>> + and instructions without trapping to EL1/EL2.
>> + Since trap overhead compromises benefits, and it's even
>> + worse in virtualization on CPU where certain IMPDEF
>> + registers and instructions are designed for EL0
>> + performance use. This assumes the kernel adds the
>> + support for Armv8.8 extension FEAT_TIDCP1.
>> +
>> amd_iommu= [HW,X86-64]
>> Pass parameters to the AMD IOMMU driver in the system.
>> Possible values are:
>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
>> index ef269a5a37e1..d12e35d799ee 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -2459,9 +2459,21 @@ static bool is_kvm_protected_mode(const struct arm64_cpu_capabilities *entry, in
>> }
>> #endif /* CONFIG_KVM */
>>
>> +static bool no_trap_el0_impdef;
>> +
>> +static int __init parse_no_trap_el0_impdef(char *p)
>> +{
>> + no_trap_el0_impdef = true;
>> + return 0;
>> +}
>> +early_param("no_trap_el0_impdef", parse_no_trap_el0_impdef);
>> +
>> static void cpu_trap_el0_impdef(const struct arm64_cpu_capabilities *__unused)
>> {
>> - sysreg_clear_set(sctlr_el1, 0, SCTLR_EL1_TIDCP);
>> + if (no_trap_el0_impdef)
>> + sysreg_clear_set(sctlr_el1, SCTLR_EL1_TIDCP, 0);
>> + else
>> + sysreg_clear_set(sctlr_el1, 0, SCTLR_EL1_TIDCP);
> An id override for TIDCP1 seems simpler rather than an early param, and consistent to how we disable other
> certain features through boot option like arm64.nosve or arm64.nompam. I see Catalin also suggested in that
> way in [1],any reason we cannot do it in that way?
Thanks for reminding me of that.
But the problem is on warm reset, SCTLR_ELx.tidcp resets to an unknown value.
This means using id_aa64mmfr1.tidcp1=0 override for TIDCP1 only skips setting
SCTLR_ELx.TIDCP to 1, it still can not guarantee the SCTLR_Ex.TIDCP is cleared.
BR,
Liao, Chang
>
> [1] https://lore.kernel.org/all/aHqamaqueuk18NyS@arm.com/
> > Thanks.
>> }
>>
>> static void cpu_enable_dit(const struct arm64_cpu_capabilities *__unused)
On 2025/10/22 17:25, Liao, Chang wrote:
> Hi, Yicong
>
> 在 2025/10/22 13:26, Yicong Yang 写道:
>> On 2025/10/21 19:54, Liao Chang wrote:
>>> Add kernel parameter to allow system-wide EL0 access to IMPDEF system
>>> regregisters and instructions without trapping to EL1/EL2. Since trap
>>> overhead will compromises benefits, and it's even worse in
>>> virtualization on CPU where certain IMPDEF registers and instructions
>>> are designed for EL0 performance use.
>>>
>>> More early discussion could be found from link below.
>>>
>>> Link: https://lore.kernel.org/all/24afb8de-622a-4865-bd8e-8e89ccfff8f4@huawei.com/
>>> Tested-by: Xinyu Zheng <zhengxinyu6@huawei.com>
>>> Signed-off-by: Liao Chang <liaochang1@huawei.com>
>>> ---
>>> Documentation/admin-guide/kernel-parameters.txt | 9 +++++++++
>>> arch/arm64/kernel/cpufeature.c | 14 +++++++++++++-
>>> 2 files changed, 22 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>>> index 5a7a83c411e9..11ffa9f7b972 100644
>>> --- a/Documentation/admin-guide/kernel-parameters.txt
>>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>>> @@ -326,6 +326,15 @@
>>> See Documentation/arch/arm64/asymmetric-32bit.rst for more
>>> information.
>>>
>>> + no_trap_el0_impdef [Arm64,EARLY]
>>> + Allow system-wide EL0 access to IMPDEF system registers
>>> + and instructions without trapping to EL1/EL2.
>>> + Since trap overhead compromises benefits, and it's even
>>> + worse in virtualization on CPU where certain IMPDEF
>>> + registers and instructions are designed for EL0
>>> + performance use. This assumes the kernel adds the
>>> + support for Armv8.8 extension FEAT_TIDCP1.
>>> +
>>> amd_iommu= [HW,X86-64]
>>> Pass parameters to the AMD IOMMU driver in the system.
>>> Possible values are:
>>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
>>> index ef269a5a37e1..d12e35d799ee 100644
>>> --- a/arch/arm64/kernel/cpufeature.c
>>> +++ b/arch/arm64/kernel/cpufeature.c
>>> @@ -2459,9 +2459,21 @@ static bool is_kvm_protected_mode(const struct arm64_cpu_capabilities *entry, in
>>> }
>>> #endif /* CONFIG_KVM */
>>>
>>> +static bool no_trap_el0_impdef;
>>> +
>>> +static int __init parse_no_trap_el0_impdef(char *p)
>>> +{
>>> + no_trap_el0_impdef = true;
>>> + return 0;
>>> +}
>>> +early_param("no_trap_el0_impdef", parse_no_trap_el0_impdef);
>>> +
>>> static void cpu_trap_el0_impdef(const struct arm64_cpu_capabilities *__unused)
>>> {
>>> - sysreg_clear_set(sctlr_el1, 0, SCTLR_EL1_TIDCP);
>>> + if (no_trap_el0_impdef)
>>> + sysreg_clear_set(sctlr_el1, SCTLR_EL1_TIDCP, 0);
>>> + else
>>> + sysreg_clear_set(sctlr_el1, 0, SCTLR_EL1_TIDCP);
>> An id override for TIDCP1 seems simpler rather than an early param, and consistent to how we disable other
>> certain features through boot option like arm64.nosve or arm64.nompam. I see Catalin also suggested in that
>> way in [1],any reason we cannot do it in that way?
> Thanks for reminding me of that.
>
> But the problem is on warm reset, SCTLR_ELx.tidcp resets to an unknown value.
> This means using id_aa64mmfr1.tidcp1=0 override for TIDCP1 only skips setting
> SCTLR_ELx.TIDCP to 1, it still can not guarantee the SCTLR_Ex.TIDCP is cleared.
it seems impossible to me. the sctlr_elx is initialized to INIT_SCTLR_EL1_MMU_ON and other bits are set
according to the feature detected. sctlr_elx is saved/restored in cpu_do_{suspend,resume} in CPU
suspend/resume so it won't be changed. so sctlr_elx.tidcp1 will remain 0 if TIDCP1 is not detected by
id_aa64mmfr1.tidcp1.
On Tue, 21 Oct 2025 12:54:28 +0100, Liao Chang <liaochang1@huawei.com> wrote: > > Add kernel parameter to allow system-wide EL0 access to IMPDEF system > regregisters and instructions without trapping to EL1/EL2. Since trap > overhead will compromises benefits, and it's even worse in > virtualization on CPU where certain IMPDEF registers and instructions > are designed for EL0 performance use. Since you mention virtualisation, I want to be clear: there is no way I will consider anything like this for KVM. KVM will always trap and UNDEF such register accesses, no matter where they come from (EL0 or EL1). Allowing such registers to be accessed from within a guest would make it impossible to context-switch or save/restore the guest correctly. You can of course do what you want in your downstream kernel or your own hypervisor, but I wanted to set the expectations on the upstream side. M. -- Without deviation from the norm, progress is not possible.
Hi Marc, 在 2025/10/21 20:25, Marc Zyngier 写道: > On Tue, 21 Oct 2025 12:54:28 +0100, > Liao Chang <liaochang1@huawei.com> wrote: >> >> Add kernel parameter to allow system-wide EL0 access to IMPDEF system >> regregisters and instructions without trapping to EL1/EL2. Since trap >> overhead will compromises benefits, and it's even worse in >> virtualization on CPU where certain IMPDEF registers and instructions >> are designed for EL0 performance use. > > Since you mention virtualisation, I want to be clear: there is no way > I will consider anything like this for KVM. KVM will always trap and > UNDEF such register accesses, no matter where they come from (EL0 or > EL1). > > Allowing such registers to be accessed from within a guest would make > it impossible to context-switch or save/restore the guest correctly. > > You can of course do what you want in your downstream kernel or your > own hypervisor, but I wanted to set the expectations on the upstream > side. Does it make sense to allow EL0 access IMPDEF without trapping for some special vendor CPUID, instead of forbidding it as the default setting on the upstream code? BR Liao, Chang > > M. >
On Wed, 22 Oct 2025 03:37:04 +0100, "Liao, Chang" <liaochang1@huawei.com> wrote: > > Hi Marc, > > 在 2025/10/21 20:25, Marc Zyngier 写道: > > On Tue, 21 Oct 2025 12:54:28 +0100, > > Liao Chang <liaochang1@huawei.com> wrote: > >> > >> Add kernel parameter to allow system-wide EL0 access to IMPDEF system > >> regregisters and instructions without trapping to EL1/EL2. Since trap > >> overhead will compromises benefits, and it's even worse in > >> virtualization on CPU where certain IMPDEF registers and instructions > >> are designed for EL0 performance use. > > > > Since you mention virtualisation, I want to be clear: there is no way > > I will consider anything like this for KVM. KVM will always trap and > > UNDEF such register accesses, no matter where they come from (EL0 or > > EL1). > > > > Allowing such registers to be accessed from within a guest would make > > it impossible to context-switch or save/restore the guest correctly. > > > > You can of course do what you want in your downstream kernel or your > > own hypervisor, but I wanted to set the expectations on the upstream > > side. > > Does it make sense to allow EL0 access IMPDEF without trapping for some > special vendor CPUID, instead of forbidding it as the default setting on > the upstream code? Let me answer your question with my own questions: How can supervisory software (kernel or hypervisor) save and restore state that it doesn't know about? How can it ensure isolation of state if there are unspecified registers that can change unspecified things behind its back? You'd need to add CPU-specific code to the kernel to make that work, and SW written to make use of these functionalities wouldn't work anywhere else. So if you end-up with a custom userspace, why should upstream care? There is zero benefit to the ecosystem. Thanks, M. -- Without deviation from the norm, progress is not possible.
在 2025/10/21 20:25, Marc Zyngier 写道: > On Tue, 21 Oct 2025 12:54:28 +0100, > Liao Chang <liaochang1@huawei.com> wrote: >> >> Add kernel parameter to allow system-wide EL0 access to IMPDEF system >> regregisters and instructions without trapping to EL1/EL2. Since trap >> overhead will compromises benefits, and it's even worse in >> virtualization on CPU where certain IMPDEF registers and instructions >> are designed for EL0 performance use. > > Since you mention virtualisation, I want to be clear: there is no way > I will consider anything like this for KVM. KVM will always trap and > UNDEF such register accesses, no matter where they come from (EL0 or > EL1). > > Allowing such registers to be accessed from within a guest would make > it impossible to context-switch or save/restore the guest correctly. You've got that right, it seems like both the guest and the host would need to save and restore those IMDDEF registers with the VM or task context.The only exception would be if the registers aren't for saving state or configuration, but instead just act as an interface to trigger a special CPU function, such as ICC_IAR1. BR, Liao, Chang > > You can of course do what you want in your downstream kernel or your > own hypervisor, but I wanted to set the expectations on the upstream > side. > > M. >
On Wed, 22 Oct 2025 02:35:02 +0100, "Liao, Chang" <liaochang1@huawei.com> wrote: > > 在 2025/10/21 20:25, Marc Zyngier 写道: > > On Tue, 21 Oct 2025 12:54:28 +0100, > > Liao Chang <liaochang1@huawei.com> wrote: > >> > >> Add kernel parameter to allow system-wide EL0 access to IMPDEF system > >> regregisters and instructions without trapping to EL1/EL2. Since trap > >> overhead will compromises benefits, and it's even worse in > >> virtualization on CPU where certain IMPDEF registers and instructions > >> are designed for EL0 performance use. > > > > Since you mention virtualisation, I want to be clear: there is no way > > I will consider anything like this for KVM. KVM will always trap and > > UNDEF such register accesses, no matter where they come from (EL0 or > > EL1). > > > > Allowing such registers to be accessed from within a guest would make > > it impossible to context-switch or save/restore the guest correctly. > > You've got that right, it seems like both the guest and the host would > need to save and restore those IMDDEF registers with the VM or task > context.The only exception would be if the registers aren't for saving > state or configuration, but instead just act as an interface to trigger > a special CPU function, such as ICC_IAR1. Funny that you mention the IAR register. Because contrary to what you seem to indicate, IAR does impact state outside of simply acknowledging an interrupt. What do you think happens to PMR, APRs, and so on? M. -- Without deviation from the norm, progress is not possible.
在 2025/10/22 16:05, Marc Zyngier 写道: > On Wed, 22 Oct 2025 02:35:02 +0100, > "Liao, Chang" <liaochang1@huawei.com> wrote: >> >> 在 2025/10/21 20:25, Marc Zyngier 写道: >>> On Tue, 21 Oct 2025 12:54:28 +0100, >>> Liao Chang <liaochang1@huawei.com> wrote: >>>> >>>> Add kernel parameter to allow system-wide EL0 access to IMPDEF system >>>> regregisters and instructions without trapping to EL1/EL2. Since trap >>>> overhead will compromises benefits, and it's even worse in >>>> virtualization on CPU where certain IMPDEF registers and instructions >>>> are designed for EL0 performance use. >>> >>> Since you mention virtualisation, I want to be clear: there is no way >>> I will consider anything like this for KVM. KVM will always trap and >>> UNDEF such register accesses, no matter where they come from (EL0 or >>> EL1). >>> >>> Allowing such registers to be accessed from within a guest would make >>> it impossible to context-switch or save/restore the guest correctly. >> >> You've got that right, it seems like both the guest and the host would >> need to save and restore those IMDDEF registers with the VM or task >> context.The only exception would be if the registers aren't for saving >> state or configuration, but instead just act as an interface to trigger >> a special CPU function, such as ICC_IAR1. > > Funny that you mention the IAR register. Because contrary to what you > seem to indicate, IAR does impact state outside of simply acknowledging > an interrupt. What do you think happens to PMR, APRs, and so on? Understood, acknowledging an interrupt will modify the active priority in APR and current running priority in RPR. BR Liao, Chang > > M. >
© 2016 - 2026 Red Hat, Inc.