[PATCH v9 6/9] x86/resctrl: Add "mba_MBps_event" file to ctrl_mon directories

Tony Luck posted 9 patches 1 year, 2 months ago
There is a newer version of this series
[PATCH v9 6/9] x86/resctrl: Add "mba_MBps_event" file to ctrl_mon directories
Posted by Tony Luck 1 year, 2 months ago
The "mba_MBps" mount option provides an alternate method to
control memory bandwidth. Instead of specifying allowable
bandwidth as a percentage of maximum possible, the user
provides a MiB/s limit value.

Historically the limit was enforced by a feedback loop from
the measure local bandwidth to adjust the memory bandwidth
allocation controls.

In preparation to allow the user to pick the memory bandwidth
monitoring event used as input to the feedback loop, provide
a file in each ctrl_mon group directory that shows the event
currently in use.

Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
---
 arch/x86/kernel/cpu/resctrl/internal.h    |  2 ++
 arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 30 +++++++++++++++++++++++
 arch/x86/kernel/cpu/resctrl/rdtgroup.c    | 10 ++++++++
 3 files changed, 42 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 485800055a7d..ce10a883ecf8 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -609,6 +609,8 @@ ssize_t rdtgroup_schemata_write(struct kernfs_open_file *of,
 				char *buf, size_t nbytes, loff_t off);
 int rdtgroup_schemata_show(struct kernfs_open_file *of,
 			   struct seq_file *s, void *v);
+int rdtgroup_mba_mbps_event_show(struct kernfs_open_file *of,
+				 struct seq_file *s, void *v);
 bool rdtgroup_cbm_overlaps(struct resctrl_schema *s, struct rdt_ctrl_domain *d,
 			   unsigned long cbm, int closid, bool exclusive);
 unsigned int rdtgroup_cbm_to_size(struct rdt_resource *r, struct rdt_ctrl_domain *d,
diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
index 200d89a64027..5fa37b4ecc7a 100644
--- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
+++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
@@ -518,6 +518,36 @@ static int smp_mon_event_count(void *arg)
 	return 0;
 }
 
+int rdtgroup_mba_mbps_event_show(struct kernfs_open_file *of,
+				 struct seq_file *s, void *v)
+{
+	struct rdtgroup *rdtgrp;
+	int ret = 0;
+
+	rdtgrp = rdtgroup_kn_lock_live(of->kn);
+
+	if (rdtgrp) {
+		switch (rdtgrp->mba_mbps_event) {
+		case QOS_L3_MBM_LOCAL_EVENT_ID:
+			seq_puts(s, "mbm_local_bytes\n");
+			break;
+		case QOS_L3_MBM_TOTAL_EVENT_ID:
+			seq_puts(s, "mbm_total_bytes\n");
+			break;
+		default:
+			pr_warn_once("Bad event %d\n", rdtgrp->mba_mbps_event);
+			ret = -EINVAL;
+			break;
+		}
+	} else {
+		ret = -ENOENT;
+	}
+
+	rdtgroup_kn_unlock(of->kn);
+
+	return ret;
+}
+
 void mon_event_read(struct rmid_read *rr, struct rdt_resource *r,
 		    struct rdt_mon_domain *d, struct rdtgroup *rdtgrp,
 		    cpumask_t *cpumask, int evtid, int first)
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index 3a89516e6f56..416e1acfad9f 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -1943,6 +1943,12 @@ static struct rftype res_common_files[] = {
 		.seq_show	= rdtgroup_schemata_show,
 		.fflags		= RFTYPE_CTRL_BASE,
 	},
+	{
+		.name		= "mba_MBps_event",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_mba_mbps_event_show,
+	},
 	{
 		.name		= "mode",
 		.mode		= 0644,
@@ -2348,6 +2354,7 @@ static int set_mba_sc(bool mba_sc)
 	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_MBA].r_resctrl;
 	u32 num_closid = resctrl_arch_get_num_closid(r);
 	struct rdt_ctrl_domain *d;
+	unsigned long fflags;
 	int i;
 
 	if (!supports_mba_mbps() || mba_sc == is_mba_sc(r))
@@ -2360,6 +2367,9 @@ static int set_mba_sc(bool mba_sc)
 			d->mbps_val[i] = MBA_MAX_MBPS;
 	}
 
+	fflags = mba_sc ? RFTYPE_CTRL_BASE | RFTYPE_MON_BASE : 0;
+	resctrl_file_fflags_init("mba_MBps_event", fflags);
+
 	return 0;
 }
 
-- 
2.47.0
Re: [PATCH v9 6/9] x86/resctrl: Add "mba_MBps_event" file to ctrl_mon directories
Posted by Reinette Chatre 1 year, 2 months ago
Hi Tony,

On 11/13/24 4:17 PM, Tony Luck wrote:
> The "mba_MBps" mount option provides an alternate method to
> control memory bandwidth. Instead of specifying allowable
> bandwidth as a percentage of maximum possible, the user
> provides a MiB/s limit value.
> 
> Historically the limit was enforced by a feedback loop from

"Historically the limit was enforced" no history needed since
this is still the case at the time of this patch.

> the measure local bandwidth to adjust the memory bandwidth
> allocation controls.

I am not sure what is meant by "a feedback loop from the measure
local bandwidth" (that was copy&pasted to next patch).

> 
> In preparation to allow the user to pick the memory bandwidth
> monitoring event used as input to the feedback loop, provide
> a file in each ctrl_mon group directory that shows the event

In the documentation the custom is to use CTRL_MON.

> currently in use.

Much better changelog. I think it will help to add a snippet that
mentions the file is only visible to user space if resctrl
was mounted with mba_MBps, and thus only visible when mba_sc is
enabled, and thus reinforcing that this maintains the contract
that rdtgrp->mba_mbps_event is only accessed when mba_sc is enabled.

> 
> Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
> Signed-off-by: Tony Luck <tony.luck@intel.com>
> ---

Reinette
Re: [PATCH v9 6/9] x86/resctrl: Add "mba_MBps_event" file to ctrl_mon directories
Posted by Luck, Tony 1 year, 2 months ago
On Tue, Nov 19, 2024 at 08:03:06PM -0800, Reinette Chatre wrote:
> Hi Tony,
> 
> On 11/13/24 4:17 PM, Tony Luck wrote:
> > The "mba_MBps" mount option provides an alternate method to
> > control memory bandwidth. Instead of specifying allowable
> > bandwidth as a percentage of maximum possible, the user
> > provides a MiB/s limit value.
> > 
> > Historically the limit was enforced by a feedback loop from
> 
> "Historically the limit was enforced" no history needed since
> this is still the case at the time of this patch.
> 
> > the measure local bandwidth to adjust the memory bandwidth
> > allocation controls.
> 
> I am not sure what is meant by "a feedback loop from the measure
> local bandwidth" (that was copy&pasted to next patch).

Dropped the whole of the "Historically" paragraph.
> 
> > 
> > In preparation to allow the user to pick the memory bandwidth
> > monitoring event used as input to the feedback loop, provide
> > a file in each ctrl_mon group directory that shows the event
> 
> In the documentation the custom is to use CTRL_MON.

Fixed here (and in several other places in the series).

> > currently in use.
> 
> Much better changelog. I think it will help to add a snippet that
> mentions the file is only visible to user space if resctrl
> was mounted with mba_MBps, and thus only visible when mba_sc is
> enabled, and thus reinforcing that this maintains the contract
> that rdtgrp->mba_mbps_event is only accessed when mba_sc is enabled.

Added text to commit message about this file only being visible
when the "mba_MBps" mount option is in use.

> > 
> > Suggested-by: Reinette Chatre <reinette.chatre@intel.com>
> > Signed-off-by: Tony Luck <tony.luck@intel.com>
> > ---
> 
> Reinette

-Tony