arm_mpam: Add KVM/arm64 and resctrl glue code

[RFC PATCH 02/38] arm64: mpam: Re-initialise MPAM regs when CPU comes online

Posted by James Morse 1 week, 3 days ago

Now that the MPAM system registers are expected to have values that change,
reprogram them based on struct task_struct when a CPU is brought online.

Previously MPAM's 'default PARTID' of 0 was used this is the PARTID that
hardware guarantees to reset. Because there are a limited number of
PARTID, this value is exposed to user space, meaning resctrl changes
to the resctrl default group would also affect kernel threads.
Instead, use the task's PARTID value for kernel work on behalf of
user-space too.

Signed-off-by: James Morse <james.morse@arm.com>
---
 arch/arm64/kernel/cpufeature.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 5ed401ff79e3..429128a181ac 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -86,6 +86,7 @@
 #include <asm/kvm_host.h>
 #include <asm/mmu.h>
 #include <asm/mmu_context.h>
+#include <asm/mpam.h>
 #include <asm/mte.h>
 #include <asm/hypervisor.h>
 #include <asm/processor.h>
@@ -2439,13 +2440,16 @@ test_has_mpam(const struct arm64_cpu_capabilities *entry, int scope)
 static void
 cpu_enable_mpam(const struct arm64_cpu_capabilities *entry)
 {
-	/*
-	 * Access by the kernel (at EL1) should use the reserved PARTID
-	 * which is configured unrestricted. This avoids priority-inversion
-	 * where latency sensitive tasks have to wait for a task that has
-	 * been throttled to release the lock.
-	 */
-	write_sysreg_s(0, SYS_MPAM1_EL1);
+	int cpu = smp_processor_id();
+	u64 regval = 0;
+
+	if (IS_ENABLED(CONFIG_MPAM))
+		regval = READ_ONCE(per_cpu(arm64_mpam_current, cpu));
+
+	write_sysreg_s(regval, SYS_MPAM1_EL1);
+	isb();
+
+	write_sysreg_s(regval, SYS_MPAM0_EL1);
 }
 
 static bool
-- 
2.39.5

Re: [RFC PATCH 02/38] arm64: mpam: Re-initialise MPAM regs when CPU comes online

Posted by Ben Horgan 6 days, 16 hours ago

Hi James,

On 12/5/25 21:58, James Morse wrote:
> Now that the MPAM system registers are expected to have values that change,
> reprogram them based on struct task_struct when a CPU is brought online.
> 
> Previously MPAM's 'default PARTID' of 0 was used this is the PARTID that
> hardware guarantees to reset. Because there are a limited number of
> PARTID, this value is exposed to user space, meaning resctrl changes
> to the resctrl default group would also affect kernel threads.
> Instead, use the task's PARTID value for kernel work on behalf of
> user-space too.
> 
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
>  arch/arm64/kernel/cpufeature.c | 18 +++++++++++-------
>  1 file changed, 11 insertions(+), 7 deletions(-)
> 
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 5ed401ff79e3..429128a181ac 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -86,6 +86,7 @@
>  #include <asm/kvm_host.h>
>  #include <asm/mmu.h>
>  #include <asm/mmu_context.h>
> +#include <asm/mpam.h>
>  #include <asm/mte.h>
>  #include <asm/hypervisor.h>
>  #include <asm/processor.h>
> @@ -2439,13 +2440,16 @@ test_has_mpam(const struct arm64_cpu_capabilities *entry, int scope)
>  static void
>  cpu_enable_mpam(const struct arm64_cpu_capabilities *entry)
>  {
> -	/*
> -	 * Access by the kernel (at EL1) should use the reserved PARTID
> -	 * which is configured unrestricted. This avoids priority-inversion
> -	 * where latency sensitive tasks have to wait for a task that has
> -	 * been throttled to release the lock.
> -	 */
> -	write_sysreg_s(0, SYS_MPAM1_EL1);
> +	int cpu = smp_processor_id();
> +	u64 regval = 0;
> +
> +	if (IS_ENABLED(CONFIG_MPAM))
> +		regval = READ_ONCE(per_cpu(arm64_mpam_current, cpu));

CONFIG_MPAM -> CONFIG_ARM64_MPAM

> +
> +	write_sysreg_s(regval, SYS_MPAM1_EL1);
> +	isb();
> +
> +	write_sysreg_s(regval, SYS_MPAM0_EL1);
>  }
>  
>  static bool

Thanks,

Ben

Re: [RFC PATCH 02/38] arm64: mpam: Re-initialise MPAM regs when CPU comes online

Posted by Ben Horgan 4 days, 20 hours ago

Hi James,

On 12/9/25 15:13, Ben Horgan wrote:
> Hi James,
> 
> On 12/5/25 21:58, James Morse wrote:
>> Now that the MPAM system registers are expected to have values that change,
>> reprogram them based on struct task_struct when a CPU is brought online.
>>
>> Previously MPAM's 'default PARTID' of 0 was used this is the PARTID that
>> hardware guarantees to reset. Because there are a limited number of
>> PARTID, this value is exposed to user space, meaning resctrl changes
>> to the resctrl default group would also affect kernel threads.
>> Instead, use the task's PARTID value for kernel work on behalf of
>> user-space too.
>>
>> Signed-off-by: James Morse <james.morse@arm.com>
>> ---
>>  arch/arm64/kernel/cpufeature.c | 18 +++++++++++-------
>>  1 file changed, 11 insertions(+), 7 deletions(-)
>>
>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
>> index 5ed401ff79e3..429128a181ac 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -86,6 +86,7 @@
>>  #include <asm/kvm_host.h>
>>  #include <asm/mmu.h>
>>  #include <asm/mmu_context.h>
>> +#include <asm/mpam.h>
>>  #include <asm/mte.h>
>>  #include <asm/hypervisor.h>
>>  #include <asm/processor.h>
>> @@ -2439,13 +2440,16 @@ test_has_mpam(const struct arm64_cpu_capabilities *entry, int scope)
>>  static void
>>  cpu_enable_mpam(const struct arm64_cpu_capabilities *entry)
>>  {
>> -	/*
>> -	 * Access by the kernel (at EL1) should use the reserved PARTID
>> -	 * which is configured unrestricted. This avoids priority-inversion
>> -	 * where latency sensitive tasks have to wait for a task that has
>> -	 * been throttled to release the lock.
>> -	 */
>> -	write_sysreg_s(0, SYS_MPAM1_EL1);
>> +	int cpu = smp_processor_id();
>> +	u64 regval = 0;
>> +
>> +	if (IS_ENABLED(CONFIG_MPAM))
>> +		regval = READ_ONCE(per_cpu(arm64_mpam_current, cpu));
> 
> CONFIG_MPAM -> CONFIG_ARM64_MPAM

Actually, this code is only run before the mpam enablement is finished,
importantly before the mpam_enabled static key is set, and so
arm64_mpam_current is still 0 for every cpu. For cpus that are brought
up after boot time this is never run.

As SYS_MPAM0_EL1 and SYS_MPAM1_EL1 are unknown out of reset we should
set them to 0 whenever a cpu comes online to make sure they initially
use PARTID 0 as that is the only one guaranteed to have sensible
defaults in the MSC. Once the mpam driver has configured the MSC we can
start setting other values.

> 
>> +
>> +	write_sysreg_s(regval, SYS_MPAM1_EL1);
>> +	isb();
>> +
>> +	write_sysreg_s(regval, SYS_MPAM0_EL1);
>>  }
>>  
>>  static bool
> 
> Thanks,
> 
> Ben
> 
> 

Thanks,

Ben

Re: [RFC PATCH 02/38] arm64: mpam: Re-initialise MPAM regs when CPU comes online

Posted by Ben Horgan 4 days, 20 hours ago

Retraction... sorry

On 12/11/25 11:23, Ben Horgan wrote:
> Hi James,
> 
> On 12/9/25 15:13, Ben Horgan wrote:
>> Hi James,
>>
>> On 12/5/25 21:58, James Morse wrote:

> 
> Actually, this code is only run before the mpam enablement is finished,
> importantly before the mpam_enabled static key is set, and so
> arm64_mpam_current is still 0 for every cpu. For cpus that are brought
> up after boot time this is never run.

Actually, actually, this is run whenever a cpu comes online, even if by
hotplug later, and so arm64_mpam_current will be 0 as need on the first
switch on and the previous value for the cpu when it has previously been
off.

> 
> As SYS_MPAM0_EL1 and SYS_MPAM1_EL1 are unknown out of reset we should
> set them to 0 whenever a cpu comes online to make sure they initially
> use PARTID 0 as that is the only one guaranteed to have sensible
> defaults in the MSC. Once the mpam driver has configured the MSC we can
> start setting other values.
> 
>>
>>> +
>>> +	write_sysreg_s(regval, SYS_MPAM1_EL1);
>>> +	isb();
>>> +
>>> +	write_sysreg_s(regval, SYS_MPAM0_EL1);
>>>  }
>>>  
>>>  static bool
>>
>> Thanks,
>>
>> Ben
>>
>>
> 
> Thanks,
> 
> Ben
> 
> 

-- 
Thanks,

Ben