Set the TCMA1 bit so that access to TTBR1 addresses with 0xf in their
tag bits will be treated as tag unchecked.

This is important to avoid unwanted tag checking on some
systems. Specifically, SCTLR_EL1.TCF can be set to indicate that no
tag check faults are desired. But the architecture doesn't guarantee
that in this case the system won't still perform tag checks.

Use TCMA1 to ensure that undesired tag checks are not performed. This
bit was already set in the KASAN case. Adding it to the non-KASAN case
prevents tag checking since all TTBR1 address will have a value of 0xf
in their tag bits.

This patch has been measured on an Ampere system to improve the following:

* Eliminate over 98% of kernel-side tag checks during "perf bench
  futex hash", as measured with "perf stat".

* Eliminate all MTE overhead (was previously a 25% performance
  penalty) from the Phoronix pts/memcached benchmark (1:10 Set:Get
  ration with 96 cores).

Reported-by: Taehyun Noh <taehyun@utexas.edu>
Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Carl Worth <carl@os.amperecomputing.com>
---
 arch/arm64/mm/proc.S | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 5d907ce3b6d3..22866b49be37 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -48,14 +48,14 @@
 #define TCR_KASAN_SW_FLAGS 0
 #endif
 
-#ifdef CONFIG_KASAN_HW_TAGS
-#define TCR_MTE_FLAGS TCR_EL1_TCMA1 | TCR_EL1_TBI1 | TCR_EL1_TBID1
-#elif defined(CONFIG_ARM64_MTE)
+#ifdef CONFIG_ARM64_MTE
 /*
  * The mte_zero_clear_page_tags() implementation uses DC GZVA, which relies on
- * TBI being enabled at EL1.
+ * TBI being enabled at EL1.  TCMA1 is needed to treat accesses with the
+ * match-all tag (0xF) as Tag Unchecked, irrespective of the SCTLR_EL1.TCF
+ * setting.
  */
-#define TCR_MTE_FLAGS TCR_EL1_TBI1 | TCR_EL1_TBID1
+#define TCR_MTE_FLAGS TCR_EL1_TCMA1 | TCR_EL1_TBI1 | TCR_EL1_TBID1
 #else
 #define TCR_MTE_FLAGS 0
 #endif

-- 
2.39.5

On 15/01/2026 11:07 pm, Carl Worth wrote:
> Set the TCMA1 bit so that access to TTBR1 addresses with 0xf in their
> tag bits will be treated as tag unchecked.
> 
> This is important to avoid unwanted tag checking on some
> systems. Specifically, SCTLR_EL1.TCF can be set to indicate that no
> tag check faults are desired. But the architecture doesn't guarantee
> that in this case the system won't still perform tag checks.
> 
> Use TCMA1 to ensure that undesired tag checks are not performed. This
> bit was already set in the KASAN case. Adding it to the non-KASAN case
> prevents tag checking since all TTBR1 address will have a value of 0xf
> in their tag bits.
> 
> This patch has been measured on an Ampere system to improve the following:
> 
> * Eliminate over 98% of kernel-side tag checks during "perf bench
>    futex hash", as measured with "perf stat".
> 
> * Eliminate all MTE overhead (was previously a 25% performance
>    penalty) from the Phoronix pts/memcached benchmark (1:10 Set:Get
>    ration with 96 cores).
> 
> Reported-by: Taehyun Noh <taehyun@utexas.edu>
> Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
> Signed-off-by: Carl Worth <carl@os.amperecomputing.com>
Fixes tag is required here so that the fix traverses to the stable 
kernels. I've not found the most appropriate commit:

973b9e373306 ("arm64: mte: move register initialization to C")
bfc62c598527 ("arm64: kasan: allow enabling in-kernel MTE")

In my opinion, bfc62c598527 should be in the fixes-by tag. At a minimum, 
the back porting should be done till 973b9e373306.

> ---
>   arch/arm64/mm/proc.S | 10 +++++-----
>   1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> index 5d907ce3b6d3..22866b49be37 100644
> --- a/arch/arm64/mm/proc.S
> +++ b/arch/arm64/mm/proc.S
> @@ -48,14 +48,14 @@
>   #define TCR_KASAN_SW_FLAGS 0
>   #endif
>   
> -#ifdef CONFIG_KASAN_HW_TAGS
> -#define TCR_MTE_FLAGS TCR_EL1_TCMA1 | TCR_EL1_TBI1 | TCR_EL1_TBID1
> -#elif defined(CONFIG_ARM64_MTE)
> +#ifdef CONFIG_ARM64_MTE
>   /*
>    * The mte_zero_clear_page_tags() implementation uses DC GZVA, which relies on
> - * TBI being enabled at EL1.
> + * TBI being enabled at EL1.  TCMA1 is needed to treat accesses with the
> + * match-all tag (0xF) as Tag Unchecked, irrespective of the SCTLR_EL1.TCF
> + * setting.
>    */
> -#define TCR_MTE_FLAGS TCR_EL1_TBI1 | TCR_EL1_TBID1
> +#define TCR_MTE_FLAGS TCR_EL1_TCMA1 | TCR_EL1_TBI1 | TCR_EL1_TBID1
>   #else
>   #define TCR_MTE_FLAGS 0
>   #endif
>

On Thu, Jan 22, 2026 at 10:23:01AM +0000, Usama Anjum wrote:
> On 15/01/2026 11:07 pm, Carl Worth wrote:
> > Set the TCMA1 bit so that access to TTBR1 addresses with 0xf in their
> > tag bits will be treated as tag unchecked.
> > 
> > This is important to avoid unwanted tag checking on some
> > systems. Specifically, SCTLR_EL1.TCF can be set to indicate that no
> > tag check faults are desired. But the architecture doesn't guarantee
> > that in this case the system won't still perform tag checks.
> > 
> > Use TCMA1 to ensure that undesired tag checks are not performed. This
> > bit was already set in the KASAN case. Adding it to the non-KASAN case
> > prevents tag checking since all TTBR1 address will have a value of 0xf
> > in their tag bits.
> > 
> > This patch has been measured on an Ampere system to improve the following:
> > 
> > * Eliminate over 98% of kernel-side tag checks during "perf bench
> >    futex hash", as measured with "perf stat".
> > 
> > * Eliminate all MTE overhead (was previously a 25% performance
> >    penalty) from the Phoronix pts/memcached benchmark (1:10 Set:Get
> >    ration with 96 cores).
> > 
> > Reported-by: Taehyun Noh <taehyun@utexas.edu>
> > Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
> > Signed-off-by: Carl Worth <carl@os.amperecomputing.com>
> 
> Fixes tag is required here so that the fix traverses to the stable kernels.
> I've not found the most appropriate commit:
> 
> 973b9e373306 ("arm64: mte: move register initialization to C")
> bfc62c598527 ("arm64: kasan: allow enabling in-kernel MTE")
> 
> In my opinion, bfc62c598527 should be in the fixes-by tag. At a minimum, the
> back porting should be done till 973b9e373306.

We can always submit it for stable backports even without a fixes tag.
It's more of a hardware implementation choice than actually fixing a
kernel bug. The previous behaviour was also correct.

-- 
Catalin

On Thu, Jan 15, 2026 at 03:07:18PM -0800, Carl Worth wrote:
> Set the TCMA1 bit so that access to TTBR1 addresses with 0xf in their
> tag bits will be treated as tag unchecked.
> 
> This is important to avoid unwanted tag checking on some
> systems. Specifically, SCTLR_EL1.TCF can be set to indicate that no
> tag check faults are desired. But the architecture doesn't guarantee
> that in this case the system won't still perform tag checks.
> 
> Use TCMA1 to ensure that undesired tag checks are not performed. This
> bit was already set in the KASAN case. Adding it to the non-KASAN case
> prevents tag checking since all TTBR1 address will have a value of 0xf
> in their tag bits.
> 
> This patch has been measured on an Ampere system to improve the following:
> 
> * Eliminate over 98% of kernel-side tag checks during "perf bench
>   futex hash", as measured with "perf stat".
> 
> * Eliminate all MTE overhead (was previously a 25% performance
>   penalty) from the Phoronix pts/memcached benchmark (1:10 Set:Get
>   ration with 96 cores).
> 
> Reported-by: Taehyun Noh <taehyun@utexas.edu>
> Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
> Signed-off-by: Carl Worth <carl@os.amperecomputing.com>

Thanks for testing an sending this.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>