xen/arch/arm/domain.c | 12 ------------ xen/arch/arm/include/asm/cpufeature.h | 1 - xen/arch/arm/include/asm/domain.h | 1 - xen/arch/arm/setup.c | 3 +-- 4 files changed, 1 insertion(+), 16 deletions(-)
It was reported that Xen no longer builds on Debian Trixie:
Assembler messages:
{standard input}:474: Error: unknown or missing system register name at operand 1 -- `msr TEECR32_EL1,x0'
{standard input}:480: Error: unknown or missing system register name at operand 1 -- `msr TEEHBR32_EL1,x0'
{standard input}:488: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEECR32_EL1'
{standard input}:494: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEEHBR32_EL1'
make[5]: *** [Rules.mk:249: arch/arm/domain.o] Error 1
This turns out to be an intentional change in bintuils. ThumbEE was dropped
from the architecture and doesn't exist in v8 (i.e. AArch64).
Xen supports v7+virt extentions so in principle we could #ifdef CONFIG_ARM_32
to keep it working, but there was apparently no use of ThumbEE outside of demo
code, so simply drop it.
Reported-by: Hans van Kranenburg <hans@knorrie.org>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
CC: Bertrand Marquis <bertrand.marquis@arm.com>
CC: Michal Orzel <michal.orzel@amd.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Hans van Kranenburg <hans@knorrie.org>
CC: Maximilian Engelhardt <maxi@daemonizer.de>
There's been no movement on this and it's urgent. This (cross)compiles for
me, but is the sum total of testing it's had.
Bintuils link: https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=8c0024ca8f6c01fc45f081f5ef11c9a6a1c24eb0
---
xen/arch/arm/domain.c | 12 ------------
xen/arch/arm/include/asm/cpufeature.h | 1 -
xen/arch/arm/include/asm/domain.h | 1 -
xen/arch/arm/setup.c | 3 +--
4 files changed, 1 insertion(+), 16 deletions(-)
diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index ab7844433597..3e32a15cac7a 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -111,12 +111,6 @@ static void ctxt_switch_from(struct vcpu *p)
p->arch.cntkctl = READ_SYSREG(CNTKCTL_EL1);
virt_timer_save(p);
- if ( is_32bit_domain(p->domain) && cpu_has_thumbee )
- {
- p->arch.teecr = READ_SYSREG(TEECR32_EL1);
- p->arch.teehbr = READ_SYSREG(TEEHBR32_EL1);
- }
-
#ifdef CONFIG_ARM_32
p->arch.joscr = READ_CP32(JOSCR);
p->arch.jmcr = READ_CP32(JMCR);
@@ -244,12 +238,6 @@ static void ctxt_switch_to(struct vcpu *n)
WRITE_SYSREG(n->arch.tpidrro_el0, TPIDRRO_EL0);
WRITE_SYSREG(n->arch.tpidr_el1, TPIDR_EL1);
- if ( is_32bit_domain(n->domain) && cpu_has_thumbee )
- {
- WRITE_SYSREG(n->arch.teecr, TEECR32_EL1);
- WRITE_SYSREG(n->arch.teehbr, TEEHBR32_EL1);
- }
-
#ifdef CONFIG_ARM_32
WRITE_CP32(n->arch.joscr, JOSCR);
WRITE_CP32(n->arch.jmcr, JMCR);
diff --git a/xen/arch/arm/include/asm/cpufeature.h b/xen/arch/arm/include/asm/cpufeature.h
index b6df18801166..bb1629546f63 100644
--- a/xen/arch/arm/include/asm/cpufeature.h
+++ b/xen/arch/arm/include/asm/cpufeature.h
@@ -29,7 +29,6 @@
#define cpu_has_thumb (boot_cpu_feature32(thumb) >= 1)
#define cpu_has_thumb2 (boot_cpu_feature32(thumb) >= 3)
#define cpu_has_jazelle (boot_cpu_feature32(jazelle) > 0)
-#define cpu_has_thumbee (boot_cpu_feature32(thumbee) == 1)
#define cpu_has_aarch32 (cpu_has_arm || cpu_has_thumb)
#ifdef CONFIG_ARM64_SVE
diff --git a/xen/arch/arm/include/asm/domain.h b/xen/arch/arm/include/asm/domain.h
index af3e168374b4..758ad807e461 100644
--- a/xen/arch/arm/include/asm/domain.h
+++ b/xen/arch/arm/include/asm/domain.h
@@ -211,7 +211,6 @@ struct arch_vcpu
register_t hcr_el2;
register_t mdcr_el2;
- uint32_t teecr, teehbr; /* ThumbEE, 32-bit guests only */
#ifdef CONFIG_ARM_32
/*
* ARMv8 only supports a trivial implementation on Jazelle when in AArch32
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 7ad870e382c2..c24e6d01c1f8 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -172,12 +172,11 @@ static void __init processor_id(void)
printk("32-bit Execution:\n");
printk(" Processor Features: %"PRIregister":%"PRIregister"\n",
system_cpuinfo.pfr32.bits[0], system_cpuinfo.pfr32.bits[1]);
- printk(" Instruction Sets:%s%s%s%s%s%s\n",
+ printk(" Instruction Sets:%s%s%s%s%s\n",
cpu_has_aarch32 ? " AArch32" : "",
cpu_has_arm ? " A32" : "",
cpu_has_thumb ? " Thumb" : "",
cpu_has_thumb2 ? " Thumb-2" : "",
- cpu_has_thumbee ? " ThumbEE" : "",
cpu_has_jazelle ? " Jazelle" : "");
printk(" Extensions:%s%s\n",
cpu_has_gentimer ? " GenericTimer" : "",
base-commit: 62bd4c2a8ee809c181d47098583270dc9db9300e
--
2.39.5Hi Andrew,
On 03/12/2025 17:16, Andrew Cooper wrote:
> It was reported that Xen no longer builds on Debian Trixie:
>
> Assembler messages:
> {standard input}:474: Error: unknown or missing system register name at operand 1 -- `msr TEECR32_EL1,x0'
> {standard input}:480: Error: unknown or missing system register name at operand 1 -- `msr TEEHBR32_EL1,x0'
> {standard input}:488: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEECR32_EL1'
> {standard input}:494: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEEHBR32_EL1'
> make[5]: *** [Rules.mk:249: arch/arm/domain.o] Error 1
>
> This turns out to be an intentional change in bintuils. ThumbEE was dropped
typo: s/binuils/binutils/
> from the architecture and doesn't exist in v8 (i.e. AArch64).
>
> Xen supports v7+virt extentions so in principle we could #ifdef CONFIG_ARM_32
typo: s/extentions/extensions/
> to keep it working, but there was apparently no use of ThumbEE outside of demo
> code, so simply drop it.
I am in favor of dropping support for ThumbEE for guest. But I am not
sure I understand this comment. Are you saying there are no processors
supporting ThumbEE where Xen would run? Asking because below, you are
removing code to context switch the ThumbEE registers. But I don't see
any code that would ensure the registers are trapping (i.e. HSTR.TEE is
set). So wouldn't this result to a cross-VM leak on those processors?
If we really don't want to support CPU where ThumbEE is available, then
we should check that "cpu_has_thumbee" is 0.
Cheers,
--
Julien Grall
On 03/12/2025 6:23 pm, Julien Grall wrote: > Hi Andrew, > > On 03/12/2025 17:16, Andrew Cooper wrote: >> to keep it working, but there was apparently no use of ThumbEE >> outside of demo >> code, so simply drop it. > > I am in favor of dropping support for ThumbEE for guest. But I am not > sure I understand this comment. There's no production use of ThumbEE known to ARM. > Are you saying there are no processors supporting ThumbEE where Xen > would run? Asking because below, you are removing code to context > switch the ThumbEE registers. But I don't see any code that would > ensure the registers are trapping (i.e. HSTR.TEE is set). So wouldn't > this result to a cross-VM leak on those processors? > > If we really don't want to support CPU where ThumbEE is available, > then we should check that "cpu_has_thumbee" is 0. The registers exist in ARM v7 only. They do not exist in ARM v8. I suppose yes this change would result in a cross-VM leak on hardware supporting ThumbEE. Can HSTR.TTEE be set unilaterally, or does it need gating on cpu_has_thumbee? Is setting HSTR.TTEE sufficient to cause an undefined instruction exception to be thrown back at a guest which goes poking? (I guess this is really "will the default do the right thing") I'll freely admit that I'm out of my depth here, but the build failure does need fixing. ~Andrew
Hi Andrew, On 03/12/2025 18:49, Andrew Cooper wrote: > On 03/12/2025 6:23 pm, Julien Grall wrote: >> Hi Andrew, >> >> On 03/12/2025 17:16, Andrew Cooper wrote: >>> to keep it working, but there was apparently no use of ThumbEE >>> outside of demo >>> code, so simply drop it. >> >> I am in favor of dropping support for ThumbEE for guest. But I am not >> sure I understand this comment. > > There's no production use of ThumbEE known to ARM. Interesting. The Arm Arm is less assertive and in fact confusing, the latest spec released in 2018 (ARM DDI 0406C.d) has the following: " From the publication of issue C.a of this manual, ARM deprecates any use of the ThumbEE instruction set. [...] ThumbEE is both the name of the instruction set and the name of the extension that provides support for that instruction set. The ThumbEE Extension is: • Required in implementations of the ARMv7-A profile. " I have also checked the TRM of the Cortex-A15 (one of the Armv7 CPU we used to have in the CI before gitlab) and it supports ThumbEE. So I think we need to additional diff you sent in this patch. > >> Are you saying there are no processors supporting ThumbEE where Xen >> would run? Asking because below, you are removing code to context >> switch the ThumbEE registers. But I don't see any code that would >> ensure the registers are trapping (i.e. HSTR.TEE is set). So wouldn't >> this result to a cross-VM leak on those processors? >> >> If we really don't want to support CPU where ThumbEE is available, >> then we should check that "cpu_has_thumbee" is 0. > > The registers exist in ARM v7 only. They do not exist in ARM v8. > > I suppose yes this change would result in a cross-VM leak on hardware > supporting ThumbEE. > > Can HSTR.TTEE be set unilaterally, or does it need gating on > cpu_has_thumbee? From my understanding of the Armv7 specification, the bit doesn't need to be gated. The Armv8 specification suggest it is RES0 so in theory it could be used in the future. So the diff you sent is correct. > > Is setting HSTR.TTEE sufficient to cause an undefined instruction > exception to be thrown back at a guest which goes poking? (I guess this > is really "will the default do the right thing") I believe so. The default behavior in Xen is to inject an undefined exception if it can't handle a trap. > > I'll freely admit that I'm out of my depth here, but the build failure > does need fixing. I agree. Do we need to include the fix in 4.19 (I think Jan asked to cut it tomorrow)? Thanks for sending the patch! Cheers, -- Julien Grall
On 03.12.2025 22:02, Julien Grall wrote: > I agree. Do we need to include the fix in 4.19 (I think Jan asked to cut > it tomorrow)? Thanks for sending the patch! Too late for 4.19.4 now, but 4.19.5 can still get it once ready. Jan
Hi Andrew and Julien, R52 which i think is one of the few Arm CPUs that some people are looking at using 32bit instructions support Thumb-2 but not ThumbEE. ThumbEE has been deprecated so I think Andrews changes (including the one that traps any ThumbEE usage which is definitely required) are Ok. Even on Cortex A15, I am not that sure that many people actually used ThumbEE. Regards Bertrand > On 3 Dec 2025, at 22:02, Julien Grall <julien@xen.org> wrote: > > Hi Andrew, > > On 03/12/2025 18:49, Andrew Cooper wrote: >> On 03/12/2025 6:23 pm, Julien Grall wrote: >>> Hi Andrew, >>> >>> On 03/12/2025 17:16, Andrew Cooper wrote: >>>> to keep it working, but there was apparently no use of ThumbEE >>>> outside of demo >>>> code, so simply drop it. >>> >>> I am in favor of dropping support for ThumbEE for guest. But I am not >>> sure I understand this comment. >> There's no production use of ThumbEE known to ARM. > > Interesting. The Arm Arm is less assertive and in fact confusing, the latest spec released in 2018 (ARM DDI 0406C.d) has the following: > > " > From the publication of issue C.a of this manual, ARM deprecates any use of the ThumbEE instruction set. > > [...] > > ThumbEE is both the name of the instruction set and the name of the extension that provides support for that > instruction set. The ThumbEE Extension is: > • Required in implementations of the ARMv7-A profile. > " > > I have also checked the TRM of the Cortex-A15 (one of the Armv7 CPU we used to have in the CI before gitlab) and it supports ThumbEE. So I think we need to additional diff you sent in this patch. > >>> Are you saying there are no processors supporting ThumbEE where Xen >>> would run? Asking because below, you are removing code to context >>> switch the ThumbEE registers. But I don't see any code that would >>> ensure the registers are trapping (i.e. HSTR.TEE is set). So wouldn't >>> this result to a cross-VM leak on those processors? >>> >>> If we really don't want to support CPU where ThumbEE is available, >>> then we should check that "cpu_has_thumbee" is 0. >> The registers exist in ARM v7 only. They do not exist in ARM v8. >> I suppose yes this change would result in a cross-VM leak on hardware >> supporting ThumbEE. >> Can HSTR.TTEE be set unilaterally, or does it need gating on >> cpu_has_thumbee? > > From my understanding of the Armv7 specification, the bit doesn't need to be gated. The Armv8 specification suggest it is RES0 so in theory it could be used in the future. So the diff you sent is correct. > >> Is setting HSTR.TTEE sufficient to cause an undefined instruction >> exception to be thrown back at a guest which goes poking? (I guess this >> is really "will the default do the right thing") > > I believe so. The default behavior in Xen is to inject an undefined exception if it can't handle a trap. > >> I'll freely admit that I'm out of my depth here, but the build failure >> does need fixing. > > I agree. Do we need to include the fix in 4.19 (I think Jan asked to cut it tomorrow)? Thanks for sending the patch! > > Cheers, > > -- > Julien Grall
On 03/12/2025 6:49 pm, Andrew Cooper wrote: > On 03/12/2025 6:23 pm, Julien Grall wrote: >> Hi Andrew, >> >> On 03/12/2025 17:16, Andrew Cooper wrote: >>> to keep it working, but there was apparently no use of ThumbEE >>> outside of demo >>> code, so simply drop it. >> I am in favor of dropping support for ThumbEE for guest. But I am not >> sure I understand this comment. > There's no production use of ThumbEE known to ARM. > >> Are you saying there are no processors supporting ThumbEE where Xen >> would run? Asking because below, you are removing code to context >> switch the ThumbEE registers. But I don't see any code that would >> ensure the registers are trapping (i.e. HSTR.TEE is set). So wouldn't >> this result to a cross-VM leak on those processors? >> >> If we really don't want to support CPU where ThumbEE is available, >> then we should check that "cpu_has_thumbee" is 0. > The registers exist in ARM v7 only. They do not exist in ARM v8. > > I suppose yes this change would result in a cross-VM leak on hardware > supporting ThumbEE. > > Can HSTR.TTEE be set unilaterally, or does it need gating on > cpu_has_thumbee? > > Is setting HSTR.TTEE sufficient to cause an undefined instruction > exception to be thrown back at a guest which goes poking? (I guess this > is really "will the default do the right thing") > > I'll freely admit that I'm out of my depth here, but the build failure > does need fixing. e.g. diff --git a/xen/arch/arm/include/asm/processor.h b/xen/arch/arm/include/asm/processor.h index 92c8bc1a3125..ec23fd098b63 100644 --- a/xen/arch/arm/include/asm/processor.h +++ b/xen/arch/arm/include/asm/processor.h @@ -411,6 +411,7 @@ /* HSTR Hyp. System Trap Register */ #define HSTR_T(x) ((_AC(1,U)<<(x))) /* Trap Cp15 c<x> */ +#define HSTR_TTEE (_AC(1,U)<<16) /* Trap ThumbEE */ /* HDCR Hyp. Debug Configuration Register */ #define HDCR_TDRA (_AC(1,U)<<11) /* Trap Debug ROM access */ diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c index 2bc3e1df0416..040c0f2e0db1 100644 --- a/xen/arch/arm/traps.c +++ b/xen/arch/arm/traps.c @@ -158,8 +158,8 @@ void init_traps(void) WRITE_SYSREG(HDCR_TDRA|HDCR_TDOSA|HDCR_TDA|HDCR_TPM|HDCR_TPMCR, MDCR_EL2); - /* Trap CP15 c15 used for implementation defined registers */ - WRITE_SYSREG(HSTR_T(15), HSTR_EL2); + /* Trap CP15 c15 used for implementation defined registers, and ThumbEE. */ + WRITE_SYSREG(HSTR_T(15) | (cpu_has_thumbee ? HSTR_TTEE : 0), HSTR_EL2); WRITE_SYSREG(get_default_cptr_flags(), CPTR_EL2);
Hi Andrew,
On 12/3/25 6:16 PM, Andrew Cooper wrote:
> It was reported that Xen no longer builds on Debian Trixie:
The FTBFS is on Debian *unstable/sid*.
The build on Debian 13 Trixie is OK.
> Assembler messages:
> {standard input}:474: Error: unknown or missing system register name at operand 1 -- `msr TEECR32_EL1,x0'
> {standard input}:480: Error: unknown or missing system register name at operand 1 -- `msr TEEHBR32_EL1,x0'
> {standard input}:488: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEECR32_EL1'
> {standard input}:494: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEEHBR32_EL1'
> make[5]: *** [Rules.mk:249: arch/arm/domain.o] Error 1
>
> This turns out to be an intentional change in bintuils. ThumbEE was dropped
> from the architecture and doesn't exist in v8 (i.e. AArch64).
>
> Xen supports v7+virt extentions so in principle we could #ifdef CONFIG_ARM_32
> to keep it working, but there was apparently no use of ThumbEE outside of demo
> code, so simply drop it.
>
> Reported-by: Hans van Kranenburg <hans@knorrie.org>
Thanks!
I have just applied the patch here and have now successfully built the
Debian Xen packages using Debian unstable/sid build chroot on the arm64
Debian VM that I have here on my work macbook.
Hans
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Stefano Stabellini <sstabellini@kernel.org>
> CC: Julien Grall <julien@xen.org>
> CC: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> CC: Bertrand Marquis <bertrand.marquis@arm.com>
> CC: Michal Orzel <michal.orzel@amd.com>
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Hans van Kranenburg <hans@knorrie.org>
> CC: Maximilian Engelhardt <maxi@daemonizer.de>
>
> There's been no movement on this and it's urgent. This (cross)compiles for
> me, but is the sum total of testing it's had.
>
> Bintuils link: https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=8c0024ca8f6c01fc45f081f5ef11c9a6a1c24eb0
> [...]
On 03/12/2025 5:47 pm, Hans van Kranenburg wrote:
> Hi Andrew,
>
> On 12/3/25 6:16 PM, Andrew Cooper wrote:
>> It was reported that Xen no longer builds on Debian Trixie:
> The FTBFS is on Debian *unstable/sid*.
>
> The build on Debian 13 Trixie is OK.
Ah ok. I can tweak that.
>
>> Assembler messages:
>> {standard input}:474: Error: unknown or missing system register name at operand 1 -- `msr TEECR32_EL1,x0'
>> {standard input}:480: Error: unknown or missing system register name at operand 1 -- `msr TEEHBR32_EL1,x0'
>> {standard input}:488: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEECR32_EL1'
>> {standard input}:494: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEEHBR32_EL1'
>> make[5]: *** [Rules.mk:249: arch/arm/domain.o] Error 1
>>
>> This turns out to be an intentional change in bintuils. ThumbEE was dropped
>> from the architecture and doesn't exist in v8 (i.e. AArch64).
>>
>> Xen supports v7+virt extentions so in principle we could #ifdef CONFIG_ARM_32
>> to keep it working, but there was apparently no use of ThumbEE outside of demo
>> code, so simply drop it.
>>
>> Reported-by: Hans van Kranenburg <hans@knorrie.org>
> Thanks!
>
> I have just applied the patch here and have now successfully built the
> Debian Xen packages using Debian unstable/sid build chroot on the arm64
> Debian VM that I have here on my work macbook.
Assuming that the maintainers are happy, can I translate this into a
Tested-by tag?
~Andrew
On 12/3/25 6:49 PM, Andrew Cooper wrote:
> On 03/12/2025 5:47 pm, Hans van Kranenburg wrote:
>> Hi Andrew,
>>
>> On 12/3/25 6:16 PM, Andrew Cooper wrote:
>>> It was reported that Xen no longer builds on Debian Trixie:
>> The FTBFS is on Debian *unstable/sid*.
>>
>> The build on Debian 13 Trixie is OK.
>
> Ah ok. I can tweak that.
>
>>
>>> Assembler messages:
>>> {standard input}:474: Error: unknown or missing system register name at operand 1 -- `msr TEECR32_EL1,x0'
>>> {standard input}:480: Error: unknown or missing system register name at operand 1 -- `msr TEEHBR32_EL1,x0'
>>> {standard input}:488: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEECR32_EL1'
>>> {standard input}:494: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEEHBR32_EL1'
>>> make[5]: *** [Rules.mk:249: arch/arm/domain.o] Error 1
>>>
>>> This turns out to be an intentional change in bintuils. ThumbEE was dropped
>>> from the architecture and doesn't exist in v8 (i.e. AArch64).
>>>
>>> Xen supports v7+virt extentions so in principle we could #ifdef CONFIG_ARM_32
>>> to keep it working, but there was apparently no use of ThumbEE outside of demo
>>> code, so simply drop it.
>>>
>>> Reported-by: Hans van Kranenburg <hans@knorrie.org>
>> Thanks!
>>
>> I have just applied the patch here and have now successfully built the
>> Debian Xen packages using Debian unstable/sid build chroot on the arm64
>> Debian VM that I have here on my work macbook.
>
> Assuming that the maintainers are happy, can I translate this into a
> Tested-by tag?
Well, it builds, but if it behaves, I do not know, since I don't have an
easy way to boot the arm64 packages here.
So, if a Build-Tested-by also exists, that one would be fine :D
Hans
© 2016 - 2025 Red Hat, Inc.