[PATCH v11 16/23] x86/resctrl: Add the functionality to unassigm MBM events

Babu Moger posted 23 patches 1 year ago
There is a newer version of this series
[PATCH v11 16/23] x86/resctrl: Add the functionality to unassigm MBM events
Posted by Babu Moger 1 year ago
The mbm_cntr_assign mode provides a limited number of hardware counters
that can be assigned to an RMID, event pair to monitor bandwidth while
assigned. If all counters are in use, the kernel will show an error
message: "Out of MBM assignable counters" when a new assignment is
requested. To make space for a new assignment, users must unassign an
already assigned counter and retry the assignment again..

Add the functionality to unassign and free the counters in the domain.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v11: Moved the functions to monitor.c.
     Renamed rdtgroup_unassign_cntr_event() to resctrl_unassign_cntr_event().
     Refactored the resctrl_unassign_cntr_event().
     Updated commit message and code comments.

v10: Patch changed again.
     Counters are managed at the domain based on the discussion.
     https://lore.kernel.org/lkml/CALPaoCj+zWq1vkHVbXYP0znJbe6Ke3PXPWjtri5AFgD9cQDCUg@mail.gmail.com/
     commit message update.

v9: Changes related to addition of new function resctrl_config_cntr().
    The removed rdtgroup_mbm_cntr_is_assigned() as it was introduced
    already.
    Text changes to take care comments.

v8: Renamed rdtgroup_mbm_cntr_is_assigned to mbm_cntr_assigned_to_domain
    Added return error handling in resctrl_arch_config_cntr().

v7: Merged rdtgroup_unassign_cntr and rdtgroup_free_cntr functions.
    Renamed rdtgroup_mbm_cntr_test() to rdtgroup_mbm_cntr_is_assigned().
    Reworded the commit log little bit.

v6: Removed mbm_cntr_free from this patch.
    Added counter test in all the domains and free if it is not assigned to
    any domains.

v5: Few name changes to match cntr_id.
    Changed the function names to rdtgroup_unassign_cntr
    More comments on commit log.

v4: Added domain specific unassign feature.
    Few name changes.

v3: Removed the static from the prototype of rdtgroup_unassign_abmc.
    The function is not called directly from user anymore. These
    changes are related to global assignment interface.

v2: No changes.
---
 arch/x86/kernel/cpu/resctrl/internal.h |  2 ++
 arch/x86/kernel/cpu/resctrl/monitor.c  | 39 ++++++++++++++++++++++++++
 2 files changed, 41 insertions(+)

diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index 547d8a4c8aba..a5b8eadc7f5c 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -729,4 +729,6 @@ int resctrl_arch_config_cntr(struct rdt_resource *r, struct rdt_mon_domain *d,
 			     u32 cntr_id, bool assign);
 int resctrl_assign_cntr_event(struct rdt_resource *r, struct rdt_mon_domain *d,
 			      struct rdtgroup *rdtgrp, enum resctrl_event_id evtid);
+int resctrl_unassign_cntr_event(struct rdt_resource *r, struct rdt_mon_domain *d,
+				struct rdtgroup *rdtgrp, enum resctrl_event_id evtid);
 #endif /* _ASM_X86_RESCTRL_INTERNAL_H */
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 127c4000a81a..b6d188d0f9b7 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -1518,3 +1518,42 @@ int resctrl_assign_cntr_event(struct rdt_resource *r, struct rdt_mon_domain *d,
 
 	return ret;
 }
+
+/*
+ * Unassign and free the counter if assigned else return success.
+ */
+static int resctrl_free_config_cntr(struct rdt_resource *r, struct rdt_mon_domain *d,
+				    struct rdtgroup *rdtgrp, enum resctrl_event_id evtid)
+{
+	int cntr_id, ret = 0;
+
+	cntr_id = mbm_cntr_get(r, d, rdtgrp, evtid);
+	if (cntr_id != -ENOENT) {
+		ret = resctrl_config_cntr(r, d, evtid, rdtgrp->mon.rmid,
+					  rdtgrp->closid, cntr_id, false);
+		if (!ret)
+			mbm_cntr_free(d, cntr_id);
+	}
+
+	return ret;
+}
+
+/*
+ * Unassign a hardware counter associated with @evtid from the domain and
+ * the group. Unassign the counters from all the domains if @d is NULL else
+ * unassign from @d.
+ */
+int resctrl_unassign_cntr_event(struct rdt_resource *r, struct rdt_mon_domain *d,
+				struct rdtgroup *rdtgrp, enum resctrl_event_id evtid)
+{
+	int ret = 0;
+
+	if (!d) {
+		list_for_each_entry(d, &r->mon_domains, hdr.list)
+			ret = resctrl_free_config_cntr(r, d, rdtgrp, evtid);
+	} else {
+		ret = resctrl_free_config_cntr(r, d, rdtgrp, evtid);
+	}
+
+	return ret;
+}
-- 
2.34.1
Re: [PATCH v11 16/23] x86/resctrl: Add the functionality to unassigm MBM events
Posted by Reinette Chatre 1 year ago
Hi Babu,

subject: unassigm -> unassign

On 1/22/25 12:20 PM, Babu Moger wrote:
> The mbm_cntr_assign mode provides a limited number of hardware counters

(now back to "limited number of hardware counters")

> that can be assigned to an RMID, event pair to monitor bandwidth while
> assigned. If all counters are in use, the kernel will show an error
> message: "Out of MBM assignable counters" when a new assignment is
> requested. To make space for a new assignment, users must unassign an

To me "kernel will show an error" implies the kernel ring buffer. Please make
the message accurate and mention that it will be in 
last_cmd_status while also considering to use -ENOSPC to help user space.

> already assigned counter and retry the assignment again..

".." -> "."

> 
> Add the functionality to unassign and free the counters in the domain.
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>

...

> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
> index 127c4000a81a..b6d188d0f9b7 100644
> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
> @@ -1518,3 +1518,42 @@ int resctrl_assign_cntr_event(struct rdt_resource *r, struct rdt_mon_domain *d,
>  
>  	return ret;
>  }
> +
> +/*
> + * Unassign and free the counter if assigned else return success.
> + */
> +static int resctrl_free_config_cntr(struct rdt_resource *r, struct rdt_mon_domain *d,
> +				    struct rdtgroup *rdtgrp, enum resctrl_event_id evtid)
> +{
> +	int cntr_id, ret = 0;
> +
> +	cntr_id = mbm_cntr_get(r, d, rdtgrp, evtid);
> +	if (cntr_id != -ENOENT) {

This can be simplified and indent level reduced with:

	cntr_id = mbm_cntr_get(r, d, rdtgrp, evtid);
	if (cntr_id < 0)
		return ret;

> +		ret = resctrl_config_cntr(r, d, evtid, rdtgrp->mon.rmid,
> +					  rdtgrp->closid, cntr_id, false);
> +		if (!ret)
> +			mbm_cntr_free(d, cntr_id);
> +	}
> +
> +	return ret;
> +}
> +
> +/*
> + * Unassign a hardware counter associated with @evtid from the domain and
> + * the group. Unassign the counters from all the domains if @d is NULL else
> + * unassign from @d.
> + */
> +int resctrl_unassign_cntr_event(struct rdt_resource *r, struct rdt_mon_domain *d,
> +				struct rdtgroup *rdtgrp, enum resctrl_event_id evtid)
> +{
> +	int ret = 0;
> +
> +	if (!d) {
> +		list_for_each_entry(d, &r->mon_domains, hdr.list)
> +			ret = resctrl_free_config_cntr(r, d, rdtgrp, evtid);

Same issue as previous patch wrt error handling.

> +	} else {
> +		ret = resctrl_free_config_cntr(r, d, rdtgrp, evtid);
> +	}
> +
> +	return ret;
> +}

Reinette
Re: [PATCH v11 16/23] x86/resctrl: Add the functionality to unassigm MBM events
Posted by Moger, Babu 1 year ago
Hi Reinette,

On 2/5/25 21:54, Reinette Chatre wrote:
> Hi Babu,
> 
> subject: unassigm -> unassign

Sure.

> 
> On 1/22/25 12:20 PM, Babu Moger wrote:
>> The mbm_cntr_assign mode provides a limited number of hardware counters
> 
> (now back to "limited number of hardware counters")

How about?

The mbm_cntr_assign mode provides "num_mbm_cntrs" number of hardware counters

> 
>> that can be assigned to an RMID, event pair to monitor bandwidth while
>> assigned. If all counters are in use, the kernel will show an error
>> message: "Out of MBM assignable counters" when a new assignment is
>> requested. To make space for a new assignment, users must unassign an
> 
> To me "kernel will show an error" implies the kernel ring buffer. Please make
> the message accurate and mention that it will be in 
> last_cmd_status while also considering to use -ENOSPC to help user space.

If all the counters are in use, the kernel will log the error message
"Unable to allocate counter in domain" in /sys/fs/resctrl/info/
last_cmd_status when a new assignment is requested. To make space for a
new assignment, users must unassign an already assigned counter and retry
the assignment again.

> 
>> already assigned counter and retry the assignment again..
> 
> ".." -> "."
> 

Sure.

>>
>> Add the functionality to unassign and free the counters in the domain.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
> 
> ...
> 
>> diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
>> index 127c4000a81a..b6d188d0f9b7 100644
>> --- a/arch/x86/kernel/cpu/resctrl/monitor.c
>> +++ b/arch/x86/kernel/cpu/resctrl/monitor.c
>> @@ -1518,3 +1518,42 @@ int resctrl_assign_cntr_event(struct rdt_resource *r, struct rdt_mon_domain *d,
>>  
>>  	return ret;
>>  }
>> +
>> +/*
>> + * Unassign and free the counter if assigned else return success.
>> + */
>> +static int resctrl_free_config_cntr(struct rdt_resource *r, struct rdt_mon_domain *d,
>> +				    struct rdtgroup *rdtgrp, enum resctrl_event_id evtid)
>> +{
>> +	int cntr_id, ret = 0;
>> +
>> +	cntr_id = mbm_cntr_get(r, d, rdtgrp, evtid);
>> +	if (cntr_id != -ENOENT) {
> 
> This can be simplified and indent level reduced with:
> 
> 	cntr_id = mbm_cntr_get(r, d, rdtgrp, evtid);
> 	if (cntr_id < 0)
> 		return ret;
> 

Sure.

>> +		ret = resctrl_config_cntr(r, d, evtid, rdtgrp->mon.rmid,
>> +					  rdtgrp->closid, cntr_id, false);
>> +		if (!ret)
>> +			mbm_cntr_free(d, cntr_id);
>> +	}
>> +
>> +	return ret;
>> +}
>> +
>> +/*
>> + * Unassign a hardware counter associated with @evtid from the domain and
>> + * the group. Unassign the counters from all the domains if @d is NULL else
>> + * unassign from @d.
>> + */
>> +int resctrl_unassign_cntr_event(struct rdt_resource *r, struct rdt_mon_domain *d,
>> +				struct rdtgroup *rdtgrp, enum resctrl_event_id evtid)
>> +{
>> +	int ret = 0;
>> +
>> +	if (!d) {
>> +		list_for_each_entry(d, &r->mon_domains, hdr.list)
>> +			ret = resctrl_free_config_cntr(r, d, rdtgrp, evtid);
> 
> Same issue as previous patch wrt error handling.

Yes.

list_for_each_entry(d, &r->mon_domains, hdr.list) {
     ret = resctrl_free_config_cntr(r, d, rdtgrp, evtid);
     if (ret)
           return ret;
}

> 
>> +	} else {
>> +		ret = resctrl_free_config_cntr(r, d, rdtgrp, evtid);
>> +	}
>> +
>> +	return ret;
>> +}
> 
> Reinette
> 

-- 
Thanks
Babu Moger
Re: [PATCH v11 16/23] x86/resctrl: Add the functionality to unassigm MBM events
Posted by Reinette Chatre 1 year ago
Hi Babu,

On 2/10/25 8:23 AM, Moger, Babu wrote:
> Hi Reinette,
> 
> On 2/5/25 21:54, Reinette Chatre wrote:
>> Hi Babu,
>>
>> subject: unassigm -> unassign
> 
> Sure.
> 
>>
>> On 1/22/25 12:20 PM, Babu Moger wrote:
>>> The mbm_cntr_assign mode provides a limited number of hardware counters
>>
>> (now back to "limited number of hardware counters")
> 
> How about?
> 
> The mbm_cntr_assign mode provides "num_mbm_cntrs" number of hardware counters

ok.

> 
>>
>>> that can be assigned to an RMID, event pair to monitor bandwidth while
>>> assigned. If all counters are in use, the kernel will show an error
>>> message: "Out of MBM assignable counters" when a new assignment is
>>> requested. To make space for a new assignment, users must unassign an
>>
>> To me "kernel will show an error" implies the kernel ring buffer. Please make
>> the message accurate and mention that it will be in 
>> last_cmd_status while also considering to use -ENOSPC to help user space.
> 
> If all the counters are in use, the kernel will log the error message
> "Unable to allocate counter in domain" in /sys/fs/resctrl/info/
> last_cmd_status when a new assignment is requested. To make space for a
> new assignment, users must unassign an already assigned counter and retry
> the assignment again.
> 

This is better, but can user space receive -ENOSPC to avoid needing to check
and parse last_cmd_status on every error?

Reinette
Re: [PATCH v11 16/23] x86/resctrl: Add the functionality to unassigm MBM events
Posted by Moger, Babu 11 months, 3 weeks ago
Hi Reinette,

On 2/10/2025 12:30 PM, Reinette Chatre wrote:
> Hi Babu,
> 
> On 2/10/25 8:23 AM, Moger, Babu wrote:
>> Hi Reinette,
>>
>> On 2/5/25 21:54, Reinette Chatre wrote:
>>> Hi Babu,
>>>
>>> subject: unassigm -> unassign
>>
>> Sure.
>>
>>>
>>> On 1/22/25 12:20 PM, Babu Moger wrote:
>>>> The mbm_cntr_assign mode provides a limited number of hardware counters
>>>
>>> (now back to "limited number of hardware counters")
>>
>> How about?
>>
>> The mbm_cntr_assign mode provides "num_mbm_cntrs" number of hardware counters
> 
> ok.
> 
>>
>>>
>>>> that can be assigned to an RMID, event pair to monitor bandwidth while
>>>> assigned. If all counters are in use, the kernel will show an error
>>>> message: "Out of MBM assignable counters" when a new assignment is
>>>> requested. To make space for a new assignment, users must unassign an
>>>
>>> To me "kernel will show an error" implies the kernel ring buffer. Please make
>>> the message accurate and mention that it will be in
>>> last_cmd_status while also considering to use -ENOSPC to help user space.
>>
>> If all the counters are in use, the kernel will log the error message
>> "Unable to allocate counter in domain" in /sys/fs/resctrl/info/
>> last_cmd_status when a new assignment is requested. To make space for a
>> new assignment, users must unassign an already assigned counter and retry
>> the assignment again.
>>
> 
> This is better, but can user space receive -ENOSPC to avoid needing to check
> and parse last_cmd_status on every error?

Yes. There was a problem in passing the error in 
resctrl_process_flags(). Took care of it now.

Thanks
Babu