[v3] x86/resctrl : Support AMD Assignable Bandwidth Monitoring Counters (ABMC)

[RFC PATCH v3 05/17] x86/resctrl: Introduce the interface to display the assignment state

Posted by Babu Moger 1 year, 10 months ago

The ABMC feature provides an option to the user to assign an RMID
to the hardware counter and monitor the bandwidth for a longer duration.
System can be in only one mode at a time (Legacy Monitor mode or ABMC
mode). By default, ABMC mode is disabled.

Provide an interface to display the monitor mode on the system.
    $cat /sys/fs/resctrl/info/L3_MON/mbm_assign
    abmc

When the feature is enabled
    $cat /sys/fs/resctrl/info/L3_MON/mbm_assign
    [abmc]

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v3: New patch to display ABMC capability.
---
 Documentation/arch/x86/resctrl.rst     |  5 +++++
 arch/x86/kernel/cpu/resctrl/monitor.c  |  4 +++-
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 17 +++++++++++++++++
 3 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/Documentation/arch/x86/resctrl.rst b/Documentation/arch/x86/resctrl.rst
index 68df7751d1f5..cd973a013525 100644
--- a/Documentation/arch/x86/resctrl.rst
+++ b/Documentation/arch/x86/resctrl.rst
@@ -257,6 +257,11 @@ with the following files:
 	    # cat /sys/fs/resctrl/info/L3_MON/mbm_local_bytes_config
 	    0=0x30;1=0x30;3=0x15;4=0x15
 
+"mbm_assign":
+	Available when assignable monitoring features are supported.
+	Reports the list of assignable features supported and the enclosed brackets
+	indicate the feature is enabled.
+
 "max_threshold_occupancy":
 		Read/write file provides the largest value (in
 		bytes) at which a previously used LLC_occupancy
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 735b449039c1..48d1957ea5a3 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -1058,8 +1058,10 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 						 RFTYPE_MON_INFO | RFTYPE_RES_CACHE);
 		}
 
-		if (resctrl_arch_has_abmc(r))
+		if (resctrl_arch_has_abmc(r)) {
 			r->mbm_assign_capable = ABMC_ASSIGN;
+			resctrl_file_fflags_init("mbm_assign", RFTYPE_MON_INFO);
+		}
 	}
 
 	l3_mon_evt_init(r);
diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index dda71fb6c10e..5ec807e8dd38 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -846,6 +846,17 @@ static int rdtgroup_rmid_show(struct kernfs_open_file *of,
 	return ret;
 }
 
+static int rdtgroup_mbm_assign_show(struct kernfs_open_file *of,
+				    struct seq_file *s, void *v)
+{
+	struct rdt_resource *r = of->kn->parent->priv;
+
+	if (r->mbm_assign_capable)
+		seq_puts(s, "abmc\n");
+
+	return 0;
+}
+
 #ifdef CONFIG_PROC_CPU_RESCTRL
 
 /*
@@ -1903,6 +1914,12 @@ static struct rftype res_common_files[] = {
 		.seq_show	= mbm_local_bytes_config_show,
 		.write		= mbm_local_bytes_config_write,
 	},
+	{
+		.name		= "mbm_assign",
+		.mode		= 0444,
+		.kf_ops		= &rdtgroup_kf_single_ops,
+		.seq_show	= rdtgroup_mbm_assign_show,
+	},
 	{
 		.name		= "cpus",
 		.mode		= 0644,
-- 
2.34.1

Re: [RFC PATCH v3 05/17] x86/resctrl: Introduce the interface to display the assignment state

Posted by Reinette Chatre 1 year, 9 months ago

Hi Babu,

On 3/28/2024 6:06 PM, Babu Moger wrote:
> The ABMC feature provides an option to the user to assign an RMID
> to the hardware counter and monitor the bandwidth for a longer duration.
> System can be in only one mode at a time (Legacy Monitor mode or ABMC
> mode). By default, ABMC mode is disabled.

"By default, ABMC mode is disabled." seems to contradict later work.

> 
> Provide an interface to display the monitor mode on the system.
>     $cat /sys/fs/resctrl/info/L3_MON/mbm_assign
>     abmc

This example seems to contradict earlier statements in two ways:
(a) it only shows one mode vs. there are two modes (legacy or ABMC)
(b) there is no active mode vs. one mode is always active.

> 
> When the feature is enabled
>     $cat /sys/fs/resctrl/info/L3_MON/mbm_assign
>     [abmc]
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> v3: New patch to display ABMC capability.
> ---
>  Documentation/arch/x86/resctrl.rst     |  5 +++++
>  arch/x86/kernel/cpu/resctrl/monitor.c  |  4 +++-
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 17 +++++++++++++++++
>  3 files changed, 25 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/arch/x86/resctrl.rst b/Documentation/arch/x86/resctrl.rst
> index 68df7751d1f5..cd973a013525 100644
> --- a/Documentation/arch/x86/resctrl.rst
> +++ b/Documentation/arch/x86/resctrl.rst
> @@ -257,6 +257,11 @@ with the following files:
>  	    # cat /sys/fs/resctrl/info/L3_MON/mbm_local_bytes_config
>  	    0=0x30;1=0x30;3=0x15;4=0x15
>  
> +"mbm_assign":
> +	Available when assignable monitoring features are supported.
> +	Reports the list of assignable features supported and the enclosed brackets
> +	indicate the feature is enabled.

"indicate the feature is enabled" -> "indicate which feature is enabled" or
"indicates the currently enabled feature" or ...?

> +
>  "max_threshold_occupancy":
>  		Read/write file provides the largest value (in
>  		bytes) at which a previously used LLC_occupancy
> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> index 735b449039c1..48d1957ea5a3 100644
> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> @@ -1058,8 +1058,10 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>  						 RFTYPE_MON_INFO | RFTYPE_RES_CACHE);
>  		}
>  
> -		if (resctrl_arch_has_abmc(r))
> +		if (resctrl_arch_has_abmc(r)) {
>  			r->mbm_assign_capable = ABMC_ASSIGN;
> +			resctrl_file_fflags_init("mbm_assign", RFTYPE_MON_INFO);

I think this will need some more thought when considering the fs/arch split.
The architecture can be expected to set r->mbm_assign_capable as above but
having the architecture meddle with the fs flags does not seem like the right
thing to do. I think that RFTYPE_MON_INFO may not be accessible to arch code
anyway.

> +		}
>  	}
>  
>  	l3_mon_evt_init(r);
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index dda71fb6c10e..5ec807e8dd38 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -846,6 +846,17 @@ static int rdtgroup_rmid_show(struct kernfs_open_file *of,
>  	return ret;
>  }
>  
> +static int rdtgroup_mbm_assign_show(struct kernfs_open_file *of,
> +				    struct seq_file *s, void *v)
> +{
> +	struct rdt_resource *r = of->kn->parent->priv;
> +
> +	if (r->mbm_assign_capable)
> +		seq_puts(s, "abmc\n");
> +
> +	return 0;
> +}

Should it print "legacy" if not mbm_assign_capable? Or actually, I think
the expectation is that this file will only be accessible if 
r->mbm_assign_capable is true ... so having that if (r->mbm_assign_capable)
check is not clear to me ... if that is false then it would be a kernel
bug, no?

> +
>  #ifdef CONFIG_PROC_CPU_RESCTRL
>  
>  /*
> @@ -1903,6 +1914,12 @@ static struct rftype res_common_files[] = {
>  		.seq_show	= mbm_local_bytes_config_show,
>  		.write		= mbm_local_bytes_config_write,
>  	},
> +	{
> +		.name		= "mbm_assign",
> +		.mode		= 0444,
> +		.kf_ops		= &rdtgroup_kf_single_ops,
> +		.seq_show	= rdtgroup_mbm_assign_show,
> +	},
>  	{
>  		.name		= "cpus",
>  		.mode		= 0644,

Reinette

Re: [RFC PATCH v3 05/17] x86/resctrl: Introduce the interface to display the assignment state

Posted by Moger, Babu 1 year, 9 months ago

Hi Reinette,

On 5/3/24 18:28, Reinette Chatre wrote:
> Hi Babu,
> 
> On 3/28/2024 6:06 PM, Babu Moger wrote:
>> The ABMC feature provides an option to the user to assign an RMID
>> to the hardware counter and monitor the bandwidth for a longer duration.
>> System can be in only one mode at a time (Legacy Monitor mode or ABMC
>> mode). By default, ABMC mode is disabled.
> 
> "By default, ABMC mode is disabled." seems to contradict later work.

Yes. Will correct it.

> 
>>
>> Provide an interface to display the monitor mode on the system.
>>     $cat /sys/fs/resctrl/info/L3_MON/mbm_assign
>>     abmc
> 
> This example seems to contradict earlier statements in two ways:
> (a) it only shows one mode vs. there are two modes (legacy or ABMC)
> (b) there is no active mode vs. one mode is always active.

I need to correct the commit messages. At this point this interface is
read-only. I will move some of this commit message to patch 15/17.

> 
>>
>> When the feature is enabled
>>     $cat /sys/fs/resctrl/info/L3_MON/mbm_assign
>>     [abmc]
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
>> v3: New patch to display ABMC capability.
>> ---
>>  Documentation/arch/x86/resctrl.rst     |  5 +++++
>>  arch/x86/kernel/cpu/resctrl/monitor.c  |  4 +++-
>>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 17 +++++++++++++++++
>>  3 files changed, 25 insertions(+), 1 deletion(-)
>>
>> diff --git a/Documentation/arch/x86/resctrl.rst b/Documentation/arch/x86/resctrl.rst
>> index 68df7751d1f5..cd973a013525 100644
>> --- a/Documentation/arch/x86/resctrl.rst
>> +++ b/Documentation/arch/x86/resctrl.rst
>> @@ -257,6 +257,11 @@ with the following files:
>>  	    # cat /sys/fs/resctrl/info/L3_MON/mbm_local_bytes_config
>>  	    0=0x30;1=0x30;3=0x15;4=0x15
>>  
>> +"mbm_assign":
>> +	Available when assignable monitoring features are supported.
>> +	Reports the list of assignable features supported and the enclosed brackets
>> +	indicate the feature is enabled.
> 
> "indicate the feature is enabled" -> "indicate which feature is enabled" or
> "indicates the currently enabled feature" or ...?

This looks good.

> 
>> +
>>  "max_threshold_occupancy":
>>  		Read/write file provides the largest value (in
>>  		bytes) at which a previously used LLC_occupancy
>> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
>> index 735b449039c1..48d1957ea5a3 100644
>> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
>> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
>> @@ -1058,8 +1058,10 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>>  						 RFTYPE_MON_INFO | RFTYPE_RES_CACHE);
>>  		}
>>  
>> -		if (resctrl_arch_has_abmc(r))
>> +		if (resctrl_arch_has_abmc(r)) {
>>  			r->mbm_assign_capable = ABMC_ASSIGN;
>> +			resctrl_file_fflags_init("mbm_assign", RFTYPE_MON_INFO);
> 
> I think this will need some more thought when considering the fs/arch split.
> The architecture can be expected to set r->mbm_assign_capable as above but
> having the architecture meddle with the fs flags does not seem like the right
> thing to do. I think that RFTYPE_MON_INFO may not be accessible to arch code
> anyway.

It is accessible to both arch and fs code per latest code.
https://lore.kernel.org/lkml/20240426150904.8854-33-Dave.Martin@arm.com/

> 
>> +		}
>>  	}
>>  
>>  	l3_mon_evt_init(r);
>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> index dda71fb6c10e..5ec807e8dd38 100644
>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> @@ -846,6 +846,17 @@ static int rdtgroup_rmid_show(struct kernfs_open_file *of,
>>  	return ret;
>>  }
>>  
>> +static int rdtgroup_mbm_assign_show(struct kernfs_open_file *of,
>> +				    struct seq_file *s, void *v)
>> +{
>> +	struct rdt_resource *r = of->kn->parent->priv;
>> +
>> +	if (r->mbm_assign_capable)
>> +		seq_puts(s, "abmc\n");
>> +
>> +	return 0;
>> +}
> 
> Should it print "legacy" if not mbm_assign_capable? Or actually, I think
> the expectation is that this file will only be accessible if 
> r->mbm_assign_capable is true ... so having that if (r->mbm_assign_capable)

Correct. I need to move code from patch 7/17 here to correct this.


> check is not clear to me ... if that is false then it would be a kernel
> bug, no?

Yes. This is not correct. I need to fix this.

> 
>> +
>>  #ifdef CONFIG_PROC_CPU_RESCTRL
>>  
>>  /*
>> @@ -1903,6 +1914,12 @@ static struct rftype res_common_files[] = {
>>  		.seq_show	= mbm_local_bytes_config_show,
>>  		.write		= mbm_local_bytes_config_write,
>>  	},
>> +	{
>> +		.name		= "mbm_assign",
>> +		.mode		= 0444,
>> +		.kf_ops		= &rdtgroup_kf_single_ops,
>> +		.seq_show	= rdtgroup_mbm_assign_show,
>> +	},
>>  	{
>>  		.name		= "cpus",
>>  		.mode		= 0644,
> 
> Reinette
> 

-- 
Thanks
Babu Moger

Re: [RFC PATCH v3 05/17] x86/resctrl: Introduce the interface to display the assignment state

Posted by Reinette Chatre 1 year, 9 months ago

Hi Babu,

On 5/7/2024 9:28 AM, Moger, Babu wrote:
> On 5/3/24 18:28, Reinette Chatre wrote:
>> On 3/28/2024 6:06 PM, Babu Moger wrote:

>>> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
>>> index 735b449039c1..48d1957ea5a3 100644
>>> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
>>> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
>>> @@ -1058,8 +1058,10 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
>>>  						 RFTYPE_MON_INFO | RFTYPE_RES_CACHE);
>>>  		}
>>>  
>>> -		if (resctrl_arch_has_abmc(r))
>>> +		if (resctrl_arch_has_abmc(r)) {
>>>  			r->mbm_assign_capable = ABMC_ASSIGN;
>>> +			resctrl_file_fflags_init("mbm_assign", RFTYPE_MON_INFO);
>>
>> I think this will need some more thought when considering the fs/arch split.
>> The architecture can be expected to set r->mbm_assign_capable as above but
>> having the architecture meddle with the fs flags does not seem like the right
>> thing to do. I think that RFTYPE_MON_INFO may not be accessible to arch code
>> anyway.
> 
> It is accessible to both arch and fs code per latest code.
> https://lore.kernel.org/lkml/20240426150904.8854-33-Dave.Martin@arm.com/

v2 of MPAM was posted without completing discussion of v1 nor addressing
all feedback.

Please read discussions surrounding #3 and #30 of v1. Note in particular suggestion
in [1] to keep RF flags internal to FS code. 

If MPAM chooses to submit new versions without completing discussions then
I have no problem letting ABMC proceed to build on what is resctrl at this time
and adding this to the pile of things MPAM needs to address during fs/arch split.

Reinette

[1] https://lore.kernel.org/lkml/b2595743-c7dc-4946-884f-ff159bc4865e@intel.com/