[RFC PATCH 14/19] x86,fs/resctrl: Add the functionality to configure PLZA

Babu Moger posted 19 patches 2 weeks, 3 days ago
[RFC PATCH 14/19] x86,fs/resctrl: Add the functionality to configure PLZA
Posted by Babu Moger 2 weeks, 3 days ago
Privilege Level Zero Association (PLZA) is configured by writing to
MSR_IA32_PQR_PLZA_ASSOC. PLZA is disabled by default on all logical
processors in the QOS Domain. System software must follow the following
sequence.

1. Set the closid, closid_en, rmid and rmid_en fields of
MSR_IA32_PQR_PLZA_ASSOC to the desired configuration on all logical
processors in the QOS Domain.

2. Set MSR_IA32_PQR_PLZA_ASSOC[PLZA_EN]=1 for
all logical processors in the QOS domain where PLZA should be enabled.

MSR_IA32_PQR_PLZA_ASSOC[PLZA_EN] may have a different value on every
logical processor in the QOS domain. The system software should perform
this as a read-modify-write to avoid changing the value of closid_en,
closid, rmid_en, and rmid fields of MSR_IA32_PQR_PLZA_ASSOC.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
 arch/x86/include/asm/resctrl.h            |  7 +++++++
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 25 +++++++++++++++++++++++
 include/linux/resctrl.h                   | 11 ++++++++++
 3 files changed, 43 insertions(+)

diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctrl.h
index 76de7d6051b7..89b38948be1a 100644
--- a/arch/x86/include/asm/resctrl.h
+++ b/arch/x86/include/asm/resctrl.h
@@ -193,6 +193,13 @@ static inline bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 ignored,
 	return READ_ONCE(tsk->rmid) == rmid;
 }
 
+static inline void resctrl_arch_set_cpu_plza(int cpu, u32 closid, u32 rmid, u32 enable)
+{
+	WRITE_ONCE(per_cpu(pqr_state.default_plza, cpu), enable);
+	WRITE_ONCE(per_cpu(pqr_state.plza_closid, cpu), closid);
+	WRITE_ONCE(per_cpu(pqr_state.plza_rmid, cpu), rmid);
+}
+
 static inline void resctrl_arch_sched_in(struct task_struct *tsk)
 {
 	if (static_branch_likely(&rdt_enable_key))
diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index b20e705606b8..79ed41bde810 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -131,3 +131,28 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable)
 
 	return 0;
 }
+
+static void resctrl_plza_set_one_amd(void *arg)
+{
+	union qos_pqr_plza_assoc *plza = arg;
+
+	wrmsrl(MSR_IA32_PQR_PLZA_ASSOC, plza->full);
+}
+
+void resctrl_arch_plza_setup(struct rdt_resource *r, u32 closid, u32 rmid)
+{
+	union qos_pqr_plza_assoc plza = { 0 };
+	struct rdt_ctrl_domain *d;
+	int cpu;
+
+	plza.split.rmid = rmid;
+	plza.split.rmid_en = 1;
+	plza.split.closid = closid;
+	plza.split.closid_en = 1;
+
+	list_for_each_entry(d, &r->ctrl_domains, hdr.list) {
+		for_each_cpu(cpu, &d->hdr.cpu_mask)
+			resctrl_arch_set_cpu_plza(cpu, closid, rmid, 0);
+		on_each_cpu_mask(&d->hdr.cpu_mask, resctrl_plza_set_one_amd, &plza, 1);
+	}
+}
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index ae252a0e6d92..ef26253ad24a 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -704,6 +704,17 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable);
  */
 bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r);
 
+/*
+ * resctrl_arch_plza_setup() - Reset all private state associated with
+ *				   all rmids and eventids.
+ * @r:		The resctrl resource.
+ * @closid:	The CLOSID to be configered for PLZA.
+ * @rmid:	The RMID to be configered for PLZA.
+ *
+ * This can be called from any CPU.
+ */
+void resctrl_arch_plza_setup(struct rdt_resource *r, u32 closid, u32 rmid);
+
 extern unsigned int resctrl_rmid_realloc_threshold;
 extern unsigned int resctrl_rmid_realloc_limit;
 
-- 
2.34.1
Re: [RFC PATCH 14/19] x86,fs/resctrl: Add the functionality to configure PLZA
Posted by Luck, Tony 1 week, 2 days ago
On Wed, Jan 21, 2026 at 03:12:52PM -0600, Babu Moger wrote:
> Privilege Level Zero Association (PLZA) is configured by writing to
> MSR_IA32_PQR_PLZA_ASSOC. PLZA is disabled by default on all logical
> processors in the QOS Domain. System software must follow the following
> sequence.
> 
> 1. Set the closid, closid_en, rmid and rmid_en fields of
> MSR_IA32_PQR_PLZA_ASSOC to the desired configuration on all logical
> processors in the QOS Domain.
> 
> 2. Set MSR_IA32_PQR_PLZA_ASSOC[PLZA_EN]=1 for
> all logical processors in the QOS domain where PLZA should be enabled.
> 
> MSR_IA32_PQR_PLZA_ASSOC[PLZA_EN] may have a different value on every
> logical processor in the QOS domain. The system software should perform
> this as a read-modify-write to avoid changing the value of closid_en,
> closid, rmid_en, and rmid fields of MSR_IA32_PQR_PLZA_ASSOC.

Architecturally this is true. But in the implementation for resctrl
there is only one PLZA group. So the CLOSID and RMID fields are
identical on every logical processor. The only changing bit is the
PLZA_EN.

The code could be simpler if you just maintained a single global
with the CLOSID/RMID bits initialized by resctrl_arch_plza_setup().

union qos_pqr_plza_assoc plza_value; // needs a better name


Change the PLZA_EN define to be

#define PLZA_EN	BIT_ULL(63)

and then the hook into the __resctrl_sched_in() becomes:


	if (static_branch_likely(&rdt_plza_enable_key)) {
		u32 plza = READ_ONCE(state->default_plza); // note, moved this inside the static branch
		tmp = READ_ONCE(tsk->plza);
		if (tmp)
			plza = tmp;

		if (plza != state->cur_plza) {
			state->cur_plza = plza;
			wrmsrq(MSR_IA32_PQR_PLZA_ASSOC,
			       (plza ? PLZA_EN : 0) | plza_value.full);
		}
	}

[Earlier e-mail about clearing the high half of MSR_IA32_PQR_PLZA_ASSOC
was wrong. My debug trace printed the wrong value. The argument to the
wrmsrl() is correct].

-Tony
Re: [RFC PATCH 14/19] x86,fs/resctrl: Add the functionality to configure PLZA
Posted by Babu Moger 1 week, 2 days ago
Hi Tony,


On 1/29/26 13:13, Luck, Tony wrote:
> On Wed, Jan 21, 2026 at 03:12:52PM -0600, Babu Moger wrote:
>> Privilege Level Zero Association (PLZA) is configured by writing to
>> MSR_IA32_PQR_PLZA_ASSOC. PLZA is disabled by default on all logical
>> processors in the QOS Domain. System software must follow the following
>> sequence.
>>
>> 1. Set the closid, closid_en, rmid and rmid_en fields of
>> MSR_IA32_PQR_PLZA_ASSOC to the desired configuration on all logical
>> processors in the QOS Domain.
>>
>> 2. Set MSR_IA32_PQR_PLZA_ASSOC[PLZA_EN]=1 for
>> all logical processors in the QOS domain where PLZA should be enabled.
>>
>> MSR_IA32_PQR_PLZA_ASSOC[PLZA_EN] may have a different value on every
>> logical processor in the QOS domain. The system software should perform
>> this as a read-modify-write to avoid changing the value of closid_en,
>> closid, rmid_en, and rmid fields of MSR_IA32_PQR_PLZA_ASSOC.
> Architecturally this is true. But in the implementation for resctrl
> there is only one PLZA group. So the CLOSID and RMID fields are
> identical on every logical processor. The only changing bit is the
> PLZA_EN.
Correct.
>
> The code could be simpler if you just maintained a single global
> with the CLOSID/RMID bits initialized by resctrl_arch_plza_setup().
>
> union qos_pqr_plza_assoc plza_value; // needs a better name
>

Yea. That is a good point.   We don't have to store CLOSID/RMID in

per-CPU state. Will do those changes in my next revision.


> Change the PLZA_EN define to be
>
> #define PLZA_EN	BIT_ULL(63)
>
> and then the hook into the __resctrl_sched_in() becomes:
>
>
> 	if (static_branch_likely(&rdt_plza_enable_key)) {
> 		u32 plza = READ_ONCE(state->default_plza); // note, moved this inside the static branch
> 		tmp = READ_ONCE(tsk->plza);
> 		if (tmp)
> 			plza = tmp;
>
> 		if (plza != state->cur_plza) {
> 			state->cur_plza = plza;
> 			wrmsrq(MSR_IA32_PQR_PLZA_ASSOC,
> 			       (plza ? PLZA_EN : 0) | plza_value.full);
> 		}
> 	}
>
> [Earlier e-mail about clearing the high half of MSR_IA32_PQR_PLZA_ASSOC
> was wrong. My debug trace printed the wrong value. The argument to the
> wrmsrl() is correct].


Got it. Thanks

Babu