[RFC PATCH v3 08/17] x86/resctrl: Initialize assignable counters bitmap

Babu Moger posted 17 patches 1 year, 10 months ago
There is a newer version of this series
[RFC PATCH v3 08/17] x86/resctrl: Initialize assignable counters bitmap
Posted by Babu Moger 1 year, 10 months ago
AMD Hardware provides a set of counters when the ABMC feature is supported.
These counters are used for assigning events to the resctrl group.

Introduce the bitmap assign_cntrs_free_map to allocate and free the
counters.

Signed-off-by: Babu Moger <babu.moger@amd.com>

---
v3: Changed the bitmap name to assign_cntrs_free_map. Removed abmc
    from the name.

v2: Changed the bitmap name to assignable_counter_free_map from
    abmc_counter_free_map.
---
 arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
index f49073c86884..2c7583e7b541 100644
--- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
+++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
@@ -186,6 +186,22 @@ bool closid_allocated(unsigned int closid)
 	return !test_bit(closid, &closid_free_map);
 }
 
+static u64 assign_cntrs_free_map;
+static u32 assign_cntrs_free_map_len;
+
+static void assign_cntrs_init(void)
+{
+	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
+
+	if (r->mbm_assign_cntrs > 64) {
+		r->mbm_assign_cntrs = 64;
+		WARN(1, "Cannot support more than 64 Assignable counters\n");
+	}
+
+	assign_cntrs_free_map = BIT_MASK(r->mbm_assign_cntrs) - 1;
+	assign_cntrs_free_map_len = r->mbm_assign_cntrs;
+}
+
 /**
  * rdtgroup_mode_by_closid - Return mode of resource group with closid
  * @closid: closid if the resource group
@@ -2459,6 +2475,9 @@ static int resctrl_abmc_setup(enum resctrl_res_level l, bool enable)
 	struct rdt_resource *r = &rdt_resources_all[l].r_resctrl;
 	struct rdt_domain *d;
 
+	/* Reset the counters bitmap */
+	assign_cntrs_init();
+
 	/* Update QOS_CFG MSR on all the CPUs in cpu_mask */
 	list_for_each_entry(d, &r->domains, list) {
 		on_each_cpu_mask(&d->cpu_mask, resctrl_abmc_msrwrite, &enable, 1);
-- 
2.34.1
Re: [RFC PATCH v3 08/17] x86/resctrl: Initialize assignable counters bitmap
Posted by Reinette Chatre 1 year, 9 months ago
Hi Babu,

On 3/28/2024 6:06 PM, Babu Moger wrote:
> AMD Hardware provides a set of counters when the ABMC feature is supported.
> These counters are used for assigning events to the resctrl group.
> 
> Introduce the bitmap assign_cntrs_free_map to allocate and free the
> counters.
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> 
> ---
> v3: Changed the bitmap name to assign_cntrs_free_map. Removed abmc
>     from the name.
> 
> v2: Changed the bitmap name to assignable_counter_free_map from
>     abmc_counter_free_map.
> ---
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> index f49073c86884..2c7583e7b541 100644
> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
> @@ -186,6 +186,22 @@ bool closid_allocated(unsigned int closid)
>  	return !test_bit(closid, &closid_free_map);
>  }
>  
> +static u64 assign_cntrs_free_map;
> +static u32 assign_cntrs_free_map_len;

Please provide summary in comments about what these globals are and how they
are used.

> +
> +static void assign_cntrs_init(void)
> +{
> +	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
> +
> +	if (r->mbm_assign_cntrs > 64) {
> +		r->mbm_assign_cntrs = 64;
> +		WARN(1, "Cannot support more than 64 Assignable counters\n");

I am a bit confused here. The configuration registers are introduced in patch #10
and if I counted right there are 5 bits for the counter id. It thus seems to me
as though there needs to be some checking during enumeration time to ensure
that all counters enumerated can be configured.

> +	}
> +
> +	assign_cntrs_free_map = BIT_MASK(r->mbm_assign_cntrs) - 1;

Please use bitmap API. For example, bitmap_fill()

> +	assign_cntrs_free_map_len = r->mbm_assign_cntrs;
> +}
> +
>  /**
>   * rdtgroup_mode_by_closid - Return mode of resource group with closid
>   * @closid: closid if the resource group
> @@ -2459,6 +2475,9 @@ static int resctrl_abmc_setup(enum resctrl_res_level l, bool enable)
>  	struct rdt_resource *r = &rdt_resources_all[l].r_resctrl;
>  	struct rdt_domain *d;
>  
> +	/* Reset the counters bitmap */
> +	assign_cntrs_init();
> +

(At this point it is unclear when resctrl_abmc_setup() is called to understand
if reset of bitmap may be appropriate. Please do expand all changelogs to help
readers along with how this implementation is intended to work.)

>  	/* Update QOS_CFG MSR on all the CPUs in cpu_mask */
>  	list_for_each_entry(d, &r->domains, list) {
>  		on_each_cpu_mask(&d->cpu_mask, resctrl_abmc_msrwrite, &enable, 1);

Reinette
Re: [RFC PATCH v3 08/17] x86/resctrl: Initialize assignable counters bitmap
Posted by Moger, Babu 1 year, 9 months ago
Hi Reinette,

On 5/3/24 18:31, Reinette Chatre wrote:
> Hi Babu,
> 
> On 3/28/2024 6:06 PM, Babu Moger wrote:
>> AMD Hardware provides a set of counters when the ABMC feature is supported.
>> These counters are used for assigning events to the resctrl group.
>>
>> Introduce the bitmap assign_cntrs_free_map to allocate and free the
>> counters.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>>
>> ---
>> v3: Changed the bitmap name to assign_cntrs_free_map. Removed abmc
>>     from the name.
>>
>> v2: Changed the bitmap name to assignable_counter_free_map from
>>     abmc_counter_free_map.
>> ---
>>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 19 +++++++++++++++++++
>>  1 file changed, 19 insertions(+)
>>
>> diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> index f49073c86884..2c7583e7b541 100644
>> --- a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> +++ b/arch/x86/kernel/cpu/resctrl/rdtgroup.c
>> @@ -186,6 +186,22 @@ bool closid_allocated(unsigned int closid)
>>  	return !test_bit(closid, &closid_free_map);
>>  }
>>  
>> +static u64 assign_cntrs_free_map;
>> +static u32 assign_cntrs_free_map_len;
> 
> Please provide summary in comments about what these globals are and how they
> are used.
> 

Sure.

>> +
>> +static void assign_cntrs_init(void)
>> +{
>> +	struct rdt_resource *r = &rdt_resources_all[RDT_RESOURCE_L3].r_resctrl;
>> +
>> +	if (r->mbm_assign_cntrs > 64) {
>> +		r->mbm_assign_cntrs = 64;
>> +		WARN(1, "Cannot support more than 64 Assignable counters\n");
> 
> I am a bit confused here. The configuration registers are introduced in patch #10
> and if I counted right there are 5 bits for the counter id. It thus seems to me
> as though there needs to be some checking during enumeration time to ensure
> that all counters enumerated can be configured.

Ok. I will remove this check here and move it to enumeration time in
rdt_get_mon_l3_config.

> 
>> +	}
>> +
>> +	assign_cntrs_free_map = BIT_MASK(r->mbm_assign_cntrs) - 1;
> 
> Please use bitmap API. For example, bitmap_fill()

Ok. Sure.

> 
>> +	assign_cntrs_free_map_len = r->mbm_assign_cntrs;
>> +}
>> +
>>  /**
>>   * rdtgroup_mode_by_closid - Return mode of resource group with closid
>>   * @closid: closid if the resource group
>> @@ -2459,6 +2475,9 @@ static int resctrl_abmc_setup(enum resctrl_res_level l, bool enable)
>>  	struct rdt_resource *r = &rdt_resources_all[l].r_resctrl;
>>  	struct rdt_domain *d;
>>  
>> +	/* Reset the counters bitmap */
>> +	assign_cntrs_init();
>> +
> 
> (At this point it is unclear when resctrl_abmc_setup() is called to understand
> if reset of bitmap may be appropriate. Please do expand all changelogs to help
> readers along with how this implementation is intended to work.)

resctrl_abmc_setup() is called either while enabling or disabling the ABMC
feature. In both these cases the bitmap assign_cntrs_free_map needs to be
reset.

Will add comments here,


> 
>>  	/* Update QOS_CFG MSR on all the CPUs in cpu_mask */
>>  	list_for_each_entry(d, &r->domains, list) {
>>  		on_each_cpu_mask(&d->cpu_mask, resctrl_abmc_msrwrite, &enable, 1);
> 
> Reinette
> 

-- 
Thanks
Babu Moger