[PATCH v15 11/32] x86,fs/resctrl: Handle events that can be read from any CPU

Tony Luck posted 32 patches 2 weeks ago
There is a newer version of this series
[PATCH v15 11/32] x86,fs/resctrl: Handle events that can be read from any CPU
Posted by Tony Luck 2 weeks ago
resctrl assumes that monitor events can only be read from a CPU in the
cpumask_t set of each domain.  This is true for x86 events accessed with an
MSR interface, but may not be true for other access methods such as MMIO.

Introduce and use flag mon_evt::any_cpu, settable by architecture, that
indicates there are no restrictions on which CPU can read that event.

Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 include/linux/resctrl.h            | 2 +-
 fs/resctrl/internal.h              | 2 ++
 arch/x86/kernel/cpu/resctrl/core.c | 6 +++---
 fs/resctrl/ctrlmondata.c           | 6 ++++++
 fs/resctrl/monitor.c               | 9 ++++++---
 5 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 79aaaabcdd3f..22c5d07fe9ff 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -412,7 +412,7 @@ u32 resctrl_arch_get_num_closid(struct rdt_resource *r);
 u32 resctrl_arch_system_num_rmid_idx(void);
 int resctrl_arch_update_domains(struct rdt_resource *r, u32 closid);
 
-void resctrl_enable_mon_event(enum resctrl_event_id eventid);
+void resctrl_enable_mon_event(enum resctrl_event_id eventid, bool any_cpu);
 
 bool resctrl_is_mon_event_enabled(enum resctrl_event_id eventid);
 
diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h
index 86cf38ab08a7..fb0b6e40d022 100644
--- a/fs/resctrl/internal.h
+++ b/fs/resctrl/internal.h
@@ -61,6 +61,7 @@ static inline struct rdt_fs_context *rdt_fc2context(struct fs_context *fc)
  *			READS_TO_REMOTE_MEM) being tracked by @evtid.
  *			Only valid if @evtid is an MBM event.
  * @configurable:	true if the event is configurable
+ * @any_cpu:		true if the event can be read from any CPU
  * @enabled:		true if the event is enabled
  */
 struct mon_evt {
@@ -69,6 +70,7 @@ struct mon_evt {
 	char			*name;
 	u32			evt_cfg;
 	bool			configurable;
+	bool			any_cpu;
 	bool			enabled;
 };
 
diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index b3a2dc56155d..bd4a98106153 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -902,15 +902,15 @@ static __init bool get_rdt_mon_resources(void)
 	bool ret = false;
 
 	if (rdt_cpu_has(X86_FEATURE_CQM_OCCUP_LLC)) {
-		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID);
+		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID, false);
 		ret = true;
 	}
 	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL)) {
-		resctrl_enable_mon_event(QOS_L3_MBM_TOTAL_EVENT_ID);
+		resctrl_enable_mon_event(QOS_L3_MBM_TOTAL_EVENT_ID, false);
 		ret = true;
 	}
 	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL)) {
-		resctrl_enable_mon_event(QOS_L3_MBM_LOCAL_EVENT_ID);
+		resctrl_enable_mon_event(QOS_L3_MBM_LOCAL_EVENT_ID, false);
 		ret = true;
 	}
 	if (rdt_cpu_has(X86_FEATURE_ABMC))
diff --git a/fs/resctrl/ctrlmondata.c b/fs/resctrl/ctrlmondata.c
index 7f9b2fed117a..2c69fcd70eeb 100644
--- a/fs/resctrl/ctrlmondata.c
+++ b/fs/resctrl/ctrlmondata.c
@@ -578,6 +578,11 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
 		}
 	}
 
+	if (evt->any_cpu) {
+		mon_event_count(rr);
+		goto out_ctx_free;
+	}
+
 	cpu = cpumask_any_housekeeping(cpumask, RESCTRL_PICK_ANY_CPU);
 
 	/*
@@ -591,6 +596,7 @@ void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
 	else
 		smp_call_on_cpu(cpu, smp_mon_event_count, rr, false);
 
+out_ctx_free:
 	if (rr->arch_mon_ctx)
 		resctrl_arch_mon_ctx_free(r, evt->evtid, rr->arch_mon_ctx);
 }
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 340b847ab397..081ff659b52c 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -518,10 +518,12 @@ static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
 {
 	switch (rr->r->rid) {
 	case RDT_RESOURCE_L3:
-		if (rr->hdr)
+		if (rr->hdr) {
+			WARN_ON_ONCE(rr->evt->any_cpu);
 			return __l3_mon_event_count(rdtgrp, rr);
-		else
+		} else {
 			return __l3_mon_event_count_sum(rdtgrp, rr);
+		}
 	default:
 		rr->err = -EINVAL;
 		return -EINVAL;
@@ -987,7 +989,7 @@ struct mon_evt mon_event_all[QOS_NUM_EVENTS] = {
 	},
 };
 
-void resctrl_enable_mon_event(enum resctrl_event_id eventid)
+void resctrl_enable_mon_event(enum resctrl_event_id eventid, bool any_cpu)
 {
 	if (WARN_ON_ONCE(eventid < QOS_FIRST_EVENT || eventid >= QOS_NUM_EVENTS))
 		return;
@@ -996,6 +998,7 @@ void resctrl_enable_mon_event(enum resctrl_event_id eventid)
 		return;
 	}
 
+	mon_event_all[eventid].any_cpu = any_cpu;
 	mon_event_all[eventid].enabled = true;
 }
 
-- 
2.51.1
Re: [PATCH v15 11/32] x86,fs/resctrl: Handle events that can be read from any CPU
Posted by Reinette Chatre 1 week, 2 days ago
Hi Tony,

On 12/4/25 12:53 PM, Tony Luck wrote:
> resctrl assumes that monitor events can only be read from a CPU in the
> cpumask_t set of each domain.  This is true for x86 events accessed with an
> MSR interface, but may not be true for other access methods such as MMIO.
> 
> Introduce and use flag mon_evt::any_cpu, settable by architecture, that
> indicates there are no restrictions on which CPU can read that event.

I propose to append (something like): "This flag is not supported by the L3
event reading that requires to be run on a CPU that belongs to the L3
domain of the event being read."

> diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
> index 340b847ab397..081ff659b52c 100644
> --- a/fs/resctrl/monitor.c
> +++ b/fs/resctrl/monitor.c
> @@ -518,10 +518,12 @@ static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
>  {
>  	switch (rr->r->rid) {
>  	case RDT_RESOURCE_L3:
> -		if (rr->hdr)
> +		if (rr->hdr) {
> +			WARN_ON_ONCE(rr->evt->any_cpu);

I appreciate that this addresses the feedback from v13 but it needs to be adjusted for the
code changes made since then. This means that this warning applies to both
__l3_mon_event_count() and __l3_mon_event_count_sum() , no?

>  			return __l3_mon_event_count(rdtgrp, rr);
> -		else
> +		} else {
>  			return __l3_mon_event_count_sum(rdtgrp, rr);
> +		}
>  	default:
>  		rr->err = -EINVAL;
>  		return -EINVAL;

Reinette
Re: [PATCH v15 11/32] x86,fs/resctrl: Handle events that can be read from any CPU
Posted by Luck, Tony 1 week, 2 days ago
On Tue, Dec 09, 2025 at 11:39:56AM -0800, Reinette Chatre wrote:
> Hi Tony,
> 
> On 12/4/25 12:53 PM, Tony Luck wrote:
> > resctrl assumes that monitor events can only be read from a CPU in the
> > cpumask_t set of each domain.  This is true for x86 events accessed with an
> > MSR interface, but may not be true for other access methods such as MMIO.
> > 
> > Introduce and use flag mon_evt::any_cpu, settable by architecture, that
> > indicates there are no restrictions on which CPU can read that event.
> 
> I propose to append (something like): "This flag is not supported by the L3
> event reading that requires to be run on a CPU that belongs to the L3
> domain of the event being read."

Perahps simpler?

This flag is not supported for RDT_RESOURCE_L3 monitoring events.

> 
> > diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
> > index 340b847ab397..081ff659b52c 100644
> > --- a/fs/resctrl/monitor.c
> > +++ b/fs/resctrl/monitor.c
> > @@ -518,10 +518,12 @@ static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
> >  {
> >  	switch (rr->r->rid) {
> >  	case RDT_RESOURCE_L3:
> > -		if (rr->hdr)
> > +		if (rr->hdr) {
> > +			WARN_ON_ONCE(rr->evt->any_cpu);
> 
> I appreciate that this addresses the feedback from v13 but it needs to be adjusted for the
> code changes made since then. This means that this warning applies to both
> __l3_mon_event_count() and __l3_mon_event_count_sum() , no?

I'l move the WARN_ON_ONCE() before the "if"

> 
> >  			return __l3_mon_event_count(rdtgrp, rr);
> > -		else
> > +		} else {
> >  			return __l3_mon_event_count_sum(rdtgrp, rr);
> > +		}
> >  	default:
> >  		rr->err = -EINVAL;
> >  		return -EINVAL;
> 
> Reinette

-Tony
Re: [PATCH v15 11/32] x86,fs/resctrl: Handle events that can be read from any CPU
Posted by Reinette Chatre 1 week, 2 days ago
Hi Tony,

On 12/9/25 12:37 PM, Luck, Tony wrote:
> On Tue, Dec 09, 2025 at 11:39:56AM -0800, Reinette Chatre wrote:
>> Hi Tony,
>>
>> On 12/4/25 12:53 PM, Tony Luck wrote:
>>> resctrl assumes that monitor events can only be read from a CPU in the
>>> cpumask_t set of each domain.  This is true for x86 events accessed with an
>>> MSR interface, but may not be true for other access methods such as MMIO.
>>>
>>> Introduce and use flag mon_evt::any_cpu, settable by architecture, that
>>> indicates there are no restrictions on which CPU can read that event.
>>
>> I propose to append (something like): "This flag is not supported by the L3
>> event reading that requires to be run on a CPU that belongs to the L3
>> domain of the event being read."
> 
> Perahps simpler?
> 
> This flag is not supported for RDT_RESOURCE_L3 monitoring events.

Simpler, but missing "why". The changelog starts by highlighting that it is x86 events
accessed with an MSR interface that can "only be read from a CPU in the cpumask_t set
of each domain" but worth highlighting is that __l3_mon_event_count()/__l3_mon_event_count_sum()
is resctrl fs code that, independent from x86, requires to be run on a CPU that belongs to
the L3 domain.

> 
>>
>>> diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
>>> index 340b847ab397..081ff659b52c 100644
>>> --- a/fs/resctrl/monitor.c
>>> +++ b/fs/resctrl/monitor.c
>>> @@ -518,10 +518,12 @@ static int __mon_event_count(struct rdtgroup *rdtgrp, struct rmid_read *rr)
>>>  {
>>>  	switch (rr->r->rid) {
>>>  	case RDT_RESOURCE_L3:
>>> -		if (rr->hdr)
>>> +		if (rr->hdr) {
>>> +			WARN_ON_ONCE(rr->evt->any_cpu);
>>
>> I appreciate that this addresses the feedback from v13 but it needs to be adjusted for the
>> code changes made since then. This means that this warning applies to both
>> __l3_mon_event_count() and __l3_mon_event_count_sum() , no?
> 
> I'l move the WARN_ON_ONCE() before the "if"

Thank you.

Reinette