[PATCH v14 09/32] x86/resctrl: Detect Assignable Bandwidth Monitoring feature details

Babu Moger posted 32 patches 3 months, 4 weeks ago
There is a newer version of this series
[PATCH v14 09/32] x86/resctrl: Detect Assignable Bandwidth Monitoring feature details
Posted by Babu Moger 3 months, 4 weeks ago
ABMC feature details are reported via CPUID Fn8000_0020_EBX_x5.
Bits Description
15:0 MAX_ABMC Maximum Supported Assignable Bandwidth
     Monitoring Counter ID + 1

The feature details are documented in APM listed below [1].
[1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
Publication # 24593 Revision 3.41 section 19.3.3.3 Assignable Bandwidth
Monitoring (ABMC).

Detect the feature and number of assignable counters supported. Also,
enable QOS_L3_MBM_TOTAL_EVENT_ID and QOS_L3_MBM_LOCAL_EVENT_ID upon
detecting the ABMC feature. The current expectation is to support
these two events by default when ABMC is enabled.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v14: Updated enumeration to support ABMC regardless of MBM total and local support.
     Updated the changelog accordingly.

v13: No changes.

v12: Resolved conflicts because of latest merge.
     Removed Reviewed-by as the patch has changed.

v11: No changes.

v10: No changes.

v9: Added Reviewed-by tag. No code changes

v8: Used GENMASK for the mask.

v7: Removed WARN_ON for num_mbm_cntrs. Decided to dynamically allocate the
    bitmap. WARN_ON is not required anymore.
    Removed redundant comments.

v6: Commit message update.
    Renamed abmc_capable to mbm_cntr_assignable.

v5: Name change num_cntrs to num_mbm_cntrs.
    Moved abmc_capable to resctrl_mon.

v4: Removed resctrl_arch_has_abmc(). Added all the code inline. We dont
    need to separate this as arch code.

v3: Removed changes related to mon_features.
    Moved rdt_cpu_has to core.c and added new function resctrl_arch_has_abmc.
    Also moved the fields mbm_assign_capable and mbm_assign_cntrs to
    rdt_resource. (James)

v2: Changed the field name to mbm_assign_capable from abmc_capable.
---
 arch/x86/kernel/cpu/resctrl/core.c    |  4 ++--
 arch/x86/kernel/cpu/resctrl/monitor.c | 11 ++++++++---
 include/linux/resctrl.h               |  4 ++++
 3 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
index 22a414802cbb..01b210febc7d 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -873,11 +873,11 @@ static __init bool get_rdt_mon_resources(void)
 		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID);
 		ret = true;
 	}
-	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL)) {
+	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL) || rdt_cpu_has(X86_FEATURE_ABMC)) {
 		resctrl_enable_mon_event(QOS_L3_MBM_TOTAL_EVENT_ID);
 		ret = true;
 	}
-	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL)) {
+	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL) || rdt_cpu_has(X86_FEATURE_ABMC)) {
 		resctrl_enable_mon_event(QOS_L3_MBM_LOCAL_EVENT_ID);
 		ret = true;
 	}
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 42a9e3cc6654..a6b9a6ba036d 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -339,6 +339,7 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 	unsigned int mbm_offset = boot_cpu_data.x86_cache_mbm_width_offset;
 	struct rdt_hw_resource *hw_res = resctrl_to_arch_res(r);
 	unsigned int threshold;
+	u32 eax, ebx, ecx, edx;
 
 	snc_nodes_per_l3_cache = snc_get_config();
 
@@ -368,14 +369,18 @@ int __init rdt_get_mon_l3_config(struct rdt_resource *r)
 	 */
 	resctrl_rmid_realloc_threshold = resctrl_arch_round_mon_val(threshold);
 
-	if (rdt_cpu_has(X86_FEATURE_BMEC)) {
-		u32 eax, ebx, ecx, edx;
-
+	if (rdt_cpu_has(X86_FEATURE_BMEC) || rdt_cpu_has(X86_FEATURE_ABMC)) {
 		/* Detect list of bandwidth sources that can be tracked */
 		cpuid_count(0x80000020, 3, &eax, &ebx, &ecx, &edx);
 		r->mon.mbm_cfg_mask = ecx & MAX_EVT_CONFIG_BITS;
 	}
 
+	if (rdt_cpu_has(X86_FEATURE_ABMC)) {
+		r->mon.mbm_cntr_assignable = true;
+		cpuid_count(0x80000020, 5, &eax, &ebx, &ecx, &edx);
+		r->mon.num_mbm_cntrs = (ebx & GENMASK(15, 0)) + 1;
+	}
+
 	r->mon_capable = true;
 
 	return 0;
diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
index 22766b8b670b..c0195498bd4a 100644
--- a/include/linux/resctrl.h
+++ b/include/linux/resctrl.h
@@ -260,10 +260,14 @@ enum resctrl_schema_fmt {
  * @num_rmid:		Number of RMIDs available.
  * @mbm_cfg_mask:	Memory transactions that can be tracked when bandwidth
  *			monitoring events are configured.
+ * @num_mbm_cntrs:	Number of assignable counters.
+ * @mbm_cntr_assignable:Is system capable of supporting counter assignment?
  */
 struct resctrl_mon {
 	int			num_rmid;
 	unsigned int		mbm_cfg_mask;
+	int			num_mbm_cntrs;
+	bool			mbm_cntr_assignable;
 };
 
 /**
-- 
2.34.1
Re: [PATCH v14 09/32] x86/resctrl: Detect Assignable Bandwidth Monitoring feature details
Posted by Reinette Chatre 3 months, 2 weeks ago
Hi Babu,

On 6/13/25 2:04 PM, Babu Moger wrote:
> ABMC feature details are reported via CPUID Fn8000_0020_EBX_x5.
> Bits Description
> 15:0 MAX_ABMC Maximum Supported Assignable Bandwidth
>      Monitoring Counter ID + 1
> 
> The feature details are documented in APM listed below [1].
> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
> Publication # 24593 Revision 3.41 section 19.3.3.3 Assignable Bandwidth
> Monitoring (ABMC).
> 
> Detect the feature and number of assignable counters supported. Also,
> enable QOS_L3_MBM_TOTAL_EVENT_ID and QOS_L3_MBM_LOCAL_EVENT_ID upon
> detecting the ABMC feature. The current expectation is to support
> these two events by default when ABMC is enabled.

"The current expectation ..." this need not be vague since this is what
this series does. Perhaps previous sentence can be:
"For backward compatibility, upon detecting the assignable counter feature,
enable the mbm_total_bytes and mbm_local_bytes events that users are
familiar with as part of original L3 MBM support." Although, when it comes to
this patch this may not be appropriate in that this is something that
resctrl fs should do, not the architecture. 


> 
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---

...

> ---
>  arch/x86/kernel/cpu/resctrl/core.c    |  4 ++--
>  arch/x86/kernel/cpu/resctrl/monitor.c | 11 ++++++++---
>  include/linux/resctrl.h               |  4 ++++
>  3 files changed, 14 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
> index 22a414802cbb..01b210febc7d 100644
> --- a/arch/x86/kernel/cpu/resctrl/core.c
> +++ b/arch/x86/kernel/cpu/resctrl/core.c
> @@ -873,11 +873,11 @@ static __init bool get_rdt_mon_resources(void)
>  		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID);
>  		ret = true;
>  	}
> -	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL)) {
> +	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL) || rdt_cpu_has(X86_FEATURE_ABMC)) {
>  		resctrl_enable_mon_event(QOS_L3_MBM_TOTAL_EVENT_ID);
>  		ret = true;
>  	}
> -	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL)) {
> +	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL) || rdt_cpu_has(X86_FEATURE_ABMC)) {
>  		resctrl_enable_mon_event(QOS_L3_MBM_LOCAL_EVENT_ID);
>  		ret = true;

This backward compatibility needs to be managed by resctrl fs, no? What do you think of
instead doing:

int resctrl_mon_resource_init(void) {

	...
	if (r->mon.mbm_cntr_assignable) {
		resctrl_enable_mon_event(QOS_L3_MBM_TOTAL_EVENT_ID);
		resctrl_enable_mon_event(QOS_L3_MBM_LOCAL_EVENT_ID);
	}
	...
}
		
There is another dependency that does not seem to be handled ... ABMC requires
properties enumerated in resctrl_cpu_detect(), but that enumeration is only
done if legacy monitoring features are supported, not ABMC. Does AMD support
enumeration CPUID.(EAX=0xF, ECX=1) if ABMC is supported but not the legacy MBM
total and local?


Reinette
Re: [PATCH v14 09/32] x86/resctrl: Detect Assignable Bandwidth Monitoring feature details
Posted by Moger, Babu 3 months, 2 weeks ago
Hi Reinette,

On 6/24/25 16:33, Reinette Chatre wrote:
> Hi Babu,
> 
> On 6/13/25 2:04 PM, Babu Moger wrote:
>> ABMC feature details are reported via CPUID Fn8000_0020_EBX_x5.
>> Bits Description
>> 15:0 MAX_ABMC Maximum Supported Assignable Bandwidth
>>      Monitoring Counter ID + 1
>>
>> The feature details are documented in APM listed below [1].
>> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
>> Publication # 24593 Revision 3.41 section 19.3.3.3 Assignable Bandwidth
>> Monitoring (ABMC).
>>
>> Detect the feature and number of assignable counters supported. Also,
>> enable QOS_L3_MBM_TOTAL_EVENT_ID and QOS_L3_MBM_LOCAL_EVENT_ID upon
>> detecting the ABMC feature. The current expectation is to support
>> these two events by default when ABMC is enabled.
> 
> "The current expectation ..." this need not be vague since this is what
> this series does. Perhaps previous sentence can be:
> "For backward compatibility, upon detecting the assignable counter feature,
> enable the mbm_total_bytes and mbm_local_bytes events that users are
> familiar with as part of original L3 MBM support." Although, when it comes to
> this patch this may not be appropriate in that this is something that
> resctrl fs should do, not the architecture. 

Sure. Added the above line. Removed the "The current expectation" line.

> 
> 
>>
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
> 
> ...
> 
>> ---
>>  arch/x86/kernel/cpu/resctrl/core.c    |  4 ++--
>>  arch/x86/kernel/cpu/resctrl/monitor.c | 11 ++++++++---
>>  include/linux/resctrl.h               |  4 ++++
>>  3 files changed, 14 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c
>> index 22a414802cbb..01b210febc7d 100644
>> --- a/arch/x86/kernel/cpu/resctrl/core.c
>> +++ b/arch/x86/kernel/cpu/resctrl/core.c
>> @@ -873,11 +873,11 @@ static __init bool get_rdt_mon_resources(void)
>>  		resctrl_enable_mon_event(QOS_L3_OCCUP_EVENT_ID);
>>  		ret = true;
>>  	}
>> -	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL)) {
>> +	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_TOTAL) || rdt_cpu_has(X86_FEATURE_ABMC)) {
>>  		resctrl_enable_mon_event(QOS_L3_MBM_TOTAL_EVENT_ID);
>>  		ret = true;
>>  	}
>> -	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL)) {
>> +	if (rdt_cpu_has(X86_FEATURE_CQM_MBM_LOCAL) || rdt_cpu_has(X86_FEATURE_ABMC)) {
>>  		resctrl_enable_mon_event(QOS_L3_MBM_LOCAL_EVENT_ID);
>>  		ret = true;
> 
> This backward compatibility needs to be managed by resctrl fs, no? What do you think of
> instead doing:

Looks good to me.

diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index dcc6c00eb362..7e816341da6a 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -924,6 +924,11 @@ int resctrl_mon_resource_init(void)
        else if (resctrl_is_mon_event_enabled(QOS_L3_MBM_TOTAL_EVENT_ID))
                mba_mbps_default_event = QOS_L3_MBM_TOTAL_EVENT_ID;

+       if (r->mon.mbm_cntr_assignable) {
+               resctrl_enable_mon_event(QOS_L3_MBM_TOTAL_EVENT_ID);
+               resctrl_enable_mon_event(QOS_L3_MBM_LOCAL_EVENT_ID);
+       }
+
        return 0;
 }


> 
> int resctrl_mon_resource_init(void) {
> 
> 	...
> 	if (r->mon.mbm_cntr_assignable) {
> 		resctrl_enable_mon_event(QOS_L3_MBM_TOTAL_EVENT_ID);
> 		resctrl_enable_mon_event(QOS_L3_MBM_LOCAL_EVENT_ID);
> 	}
> 	...
> }
> 		
> There is another dependency that does not seem to be handled ... ABMC requires
> properties enumerated in resctrl_cpu_detect(), but that enumeration is only
> done if legacy monitoring features are supported, not ABMC. Does AMD support
> enumeration CPUID.(EAX=0xF, ECX=1) if ABMC is supported but not the legacy MBM
> total and local?

Yes. The CPUID.(EAX=0xF, ECX=1) is kind of building block. I would think
it will always be supported.

Added this check now.

diff --git a/arch/x86/kernel/cpu/resctrl/core.c
b/arch/x86/kernel/cpu/resctrl/core.c
index 22a414802cbb..e9a8c4778d22 100644
--- a/arch/x86/kernel/cpu/resctrl/core.c
+++ b/arch/x86/kernel/cpu/resctrl/core.c
@@ -988,7 +988,8 @@ void resctrl_cpu_detect(struct cpuinfo_x86 *c)

        if (cpu_has(c, X86_FEATURE_CQM_OCCUP_LLC) ||
            cpu_has(c, X86_FEATURE_CQM_MBM_TOTAL) ||
-           cpu_has(c, X86_FEATURE_CQM_MBM_LOCAL)) {
+           cpu_has(c, X86_FEATURE_CQM_MBM_LOCAL) ||
+           cpu_has(c, X86_FEATURE_ABMC)) {
                u32 eax, ebx, ecx, edx;

                /* QoS sub-leaf, EAX=0Fh, ECX=1 */


-- 
Thanks
Babu Moger