[v2] Fix a couple of corner cases in feec() when using uclamp_max

[PATCH v2 2/3] sched/uclamp: Ignore (util == 0) optimization in feec() when p_util_max = 0

Posted by Qais Yousef 2 years, 7 months ago

find_energy_efficient_cpu() bails out early if effective util of the
task is 0. When uclamp is being used, this could lead to wrong decisions
when uclamp_max is set to 0. Cater for that.

Fixes: d81304bc6193 ("sched/uclamp: Cater for uclamp in find_energy_efficient_cpu()'s early exit condition")
Signed-off-by: Qais Yousef <qyousef@layalina.io>
---
 kernel/sched/fair.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 7a21ee74139f..a8c3d92ff3f6 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7374,7 +7374,7 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu)
 	target = prev_cpu;
 
 	sync_entity_load_avg(&p->se);
-	if (!uclamp_task_util(p, p_util_min, p_util_max))
+	if (!uclamp_task_util(p, p_util_min, p_util_max) && p_util_max != 0)
 		goto unlock;
 
 	eenv_task_busy_time(&eenv, p, prev_cpu);
-- 
2.25.1

Re: [PATCH v2 2/3] sched/uclamp: Ignore (util == 0) optimization in feec() when p_util_max = 0

Posted by Vincent Guittot 2 years, 7 months ago

On Sun, 5 Feb 2023 at 23:43, Qais Yousef <qyousef@layalina.io> wrote:
>
> find_energy_efficient_cpu() bails out early if effective util of the
> task is 0. When uclamp is being used, this could lead to wrong decisions
> when uclamp_max is set to 0. Cater for that.
>
> Fixes: d81304bc6193 ("sched/uclamp: Cater for uclamp in find_energy_efficient_cpu()'s early exit condition")
> Signed-off-by: Qais Yousef <qyousef@layalina.io>
> ---
>  kernel/sched/fair.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 7a21ee74139f..a8c3d92ff3f6 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -7374,7 +7374,7 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu)
>         target = prev_cpu;
>
>         sync_entity_load_avg(&p->se);
> -       if (!uclamp_task_util(p, p_util_min, p_util_max))
> +       if (!uclamp_task_util(p, p_util_min, p_util_max) && p_util_max != 0)

The below should do the same without testing twice p_util_max:
uclamp_task_util(p, p_util_min, ULONG_MAX)


>                 goto unlock;
>
>         eenv_task_busy_time(&eenv, p, prev_cpu);
> --
> 2.25.1
>

Re: [PATCH v2 2/3] sched/uclamp: Ignore (util == 0) optimization in feec() when p_util_max = 0

Posted by Dietmar Eggemann 2 years, 7 months ago

On 07/02/2023 11:04, Vincent Guittot wrote:
> On Sun, 5 Feb 2023 at 23:43, Qais Yousef <qyousef@layalina.io> wrote:
>>
>> find_energy_efficient_cpu() bails out early if effective util of the
>> task is 0. When uclamp is being used, this could lead to wrong decisions
>> when uclamp_max is set to 0. Cater for that.

IMHO this needs a little bit more explanation. Someone could argue that
'util > 0, uclamp_min=0, uclamp_max=0' is a valid setup for a task which
should let it appear as a task with 0 util (capped to 0).

>> Fixes: d81304bc6193 ("sched/uclamp: Cater for uclamp in find_energy_efficient_cpu()'s early exit condition")
>> Signed-off-by: Qais Yousef <qyousef@layalina.io>
>> ---
>>  kernel/sched/fair.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>> index 7a21ee74139f..a8c3d92ff3f6 100644
>> --- a/kernel/sched/fair.c
>> +++ b/kernel/sched/fair.c
>> @@ -7374,7 +7374,7 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu)
>>         target = prev_cpu;
>>
>>         sync_entity_load_avg(&p->se);
>> -       if (!uclamp_task_util(p, p_util_min, p_util_max))
>> +       if (!uclamp_task_util(p, p_util_min, p_util_max) && p_util_max != 0)
> 
> The below should do the same without testing twice p_util_max:
> uclamp_task_util(p, p_util_min, ULONG_MAX)

Since uclamp_task_util() is only used here and we don't want to test for
capping to 0 anymore, why not just get rid of this function and use:

  !(task_util_est(p) || p_util_min)

[...]

Re: [PATCH v2 2/3] sched/uclamp: Ignore (util == 0) optimization in feec() when p_util_max = 0

Posted by Qais Yousef 2 years, 7 months ago

On 02/08/23 12:52, Dietmar Eggemann wrote:
> On 07/02/2023 11:04, Vincent Guittot wrote:
> > On Sun, 5 Feb 2023 at 23:43, Qais Yousef <qyousef@layalina.io> wrote:
> >>
> >> find_energy_efficient_cpu() bails out early if effective util of the
> >> task is 0. When uclamp is being used, this could lead to wrong decisions
> >> when uclamp_max is set to 0. Cater for that.
> 
> IMHO this needs a little bit more explanation. Someone could argue that
> 'util > 0, uclamp_min=0, uclamp_max=0' is a valid setup for a task which
> should let it appear as a task with 0 util (capped to 0).

You want me to explain the purpose of the optimization then?

The optimization skips energy calculation when util is 0 because the delta will
be 0. But when uclamp_max = 0 util is not really 0 - consequently  the delta
will not be 0.

Would such an explanation clarify things better?

> 
> >> Fixes: d81304bc6193 ("sched/uclamp: Cater for uclamp in find_energy_efficient_cpu()'s early exit condition")
> >> Signed-off-by: Qais Yousef <qyousef@layalina.io>
> >> ---
> >>  kernel/sched/fair.c | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> >> index 7a21ee74139f..a8c3d92ff3f6 100644
> >> --- a/kernel/sched/fair.c
> >> +++ b/kernel/sched/fair.c
> >> @@ -7374,7 +7374,7 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu)
> >>         target = prev_cpu;
> >>
> >>         sync_entity_load_avg(&p->se);
> >> -       if (!uclamp_task_util(p, p_util_min, p_util_max))
> >> +       if (!uclamp_task_util(p, p_util_min, p_util_max) && p_util_max != 0)
> > 
> > The below should do the same without testing twice p_util_max:
> > uclamp_task_util(p, p_util_min, ULONG_MAX)
> 
> Since uclamp_task_util() is only used here and we don't want to test for
> capping to 0 anymore, why not just get rid of this function and use:
> 
>   !(task_util_est(p) || p_util_min)

That would be better, yes!

Question for you and Vincent. Do we really want this optimization? I started
with removing it - then erred on the conservative side and kept it.

I don't know how often we hit this case and I didn't see any benchmark run to
be able to verify anything when I looked at the history.

It seems helpful in theory - but why we save something if we ignore 0 but not
1 which I suspect will not produce a significant delta either.

I don't mind keeping it - but I think worth thinking if it is really adding
much.

Cheers

--
Qais Yousef

Re: [PATCH v2 2/3] sched/uclamp: Ignore (util == 0) optimization in feec() when p_util_max = 0

Posted by Dietmar Eggemann 2 years, 7 months ago

On 11/02/2023 19:01, Qais Yousef wrote:
> On 02/08/23 12:52, Dietmar Eggemann wrote:
>> On 07/02/2023 11:04, Vincent Guittot wrote:
>>> On Sun, 5 Feb 2023 at 23:43, Qais Yousef <qyousef@layalina.io> wrote:
>>>>
>>>> find_energy_efficient_cpu() bails out early if effective util of the
>>>> task is 0. When uclamp is being used, this could lead to wrong decisions
>>>> when uclamp_max is set to 0. Cater for that.
>>
>> IMHO this needs a little bit more explanation. Someone could argue that
>> 'util > 0, uclamp_min=0, uclamp_max=0' is a valid setup for a task which
>> should let it appear as a task with 0 util (capped to 0).
> 
> You want me to explain the purpose of the optimization then?
> 
> The optimization skips energy calculation when util is 0 because the delta will
> be 0. But when uclamp_max = 0 util is not really 0 - consequently  the delta

I would say:

s/really/necessarily
s/delta/energy delta

> will not be 0.
> 
> Would such an explanation clarify things better?

Yes. It key to understand that there is a 2-step process. First,
admittance to a possible target (util and uclamp) and second, energy
diff based target-selection (util).

>>>> Fixes: d81304bc6193 ("sched/uclamp: Cater for uclamp in find_energy_efficient_cpu()'s early exit condition")
>>>> Signed-off-by: Qais Yousef <qyousef@layalina.io>
>>>> ---
>>>>  kernel/sched/fair.c | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>>>> index 7a21ee74139f..a8c3d92ff3f6 100644
>>>> --- a/kernel/sched/fair.c
>>>> +++ b/kernel/sched/fair.c
>>>> @@ -7374,7 +7374,7 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu)
>>>>         target = prev_cpu;
>>>>
>>>>         sync_entity_load_avg(&p->se);
>>>> -       if (!uclamp_task_util(p, p_util_min, p_util_max))
>>>> +       if (!uclamp_task_util(p, p_util_min, p_util_max) && p_util_max != 0)
>>>
>>> The below should do the same without testing twice p_util_max:
>>> uclamp_task_util(p, p_util_min, ULONG_MAX)
>>
>> Since uclamp_task_util() is only used here and we don't want to test for
>> capping to 0 anymore, why not just get rid of this function and use:
>>
>>   !(task_util_est(p) || p_util_min)
> 
> That would be better, yes!
> 
> Question for you and Vincent. Do we really want this optimization? I started
> with removing it - then erred on the conservative side and kept it.

Hard to say ... at least we know that util=0 will have absolutely no
effect on task placement. So we can spare the heavy EAS algorithm in
this case for sure.

> I don't know how often we hit this case and I didn't see any benchmark run to
> be able to verify anything when I looked at the history.

There are very few EAS wakeups with `task_util_est(p) = 0`. Probably not
relevant.

> It seems helpful in theory - but why we save something if we ignore 0 but not
> 1 which I suspect will not produce a significant delta either.

True, it's hard to find the real line of significance here.

> I don't mind keeping it - but I think worth thinking if it is really adding
> much.

I would keep it and just remove uclamp_task_util(). We still have a lot
of uclamp/util related functions, we should try to keep the number as
low as possible. Just checked it, this check has been there from the
beginning of EAS.

Re: [PATCH v2 2/3] sched/uclamp: Ignore (util == 0) optimization in feec() when p_util_max = 0

Posted by Vincent Guittot 2 years, 6 months ago

On Tue, 14 Feb 2023 at 13:47, Dietmar Eggemann <dietmar.eggemann@arm.com> wrote:
>
> On 11/02/2023 19:01, Qais Yousef wrote:
> > On 02/08/23 12:52, Dietmar Eggemann wrote:
> >> On 07/02/2023 11:04, Vincent Guittot wrote:
> >>> On Sun, 5 Feb 2023 at 23:43, Qais Yousef <qyousef@layalina.io> wrote:
> >>>>
> >>>> find_energy_efficient_cpu() bails out early if effective util of the
> >>>> task is 0. When uclamp is being used, this could lead to wrong decisions
> >>>> when uclamp_max is set to 0. Cater for that.
> >>
> >> IMHO this needs a little bit more explanation. Someone could argue that
> >> 'util > 0, uclamp_min=0, uclamp_max=0' is a valid setup for a task which
> >> should let it appear as a task with 0 util (capped to 0).
> >
> > You want me to explain the purpose of the optimization then?
> >
> > The optimization skips energy calculation when util is 0 because the delta will
> > be 0. But when uclamp_max = 0 util is not really 0 - consequently  the delta
>
> I would say:
>
> s/really/necessarily
> s/delta/energy delta
>
> > will not be 0.
> >
> > Would such an explanation clarify things better?
>
> Yes. It key to understand that there is a 2-step process. First,
> admittance to a possible target (util and uclamp) and second, energy
> diff based target-selection (util).
>
> >>>> Fixes: d81304bc6193 ("sched/uclamp: Cater for uclamp in find_energy_efficient_cpu()'s early exit condition")
> >>>> Signed-off-by: Qais Yousef <qyousef@layalina.io>
> >>>> ---
> >>>>  kernel/sched/fair.c | 2 +-
> >>>>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>>>
> >>>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> >>>> index 7a21ee74139f..a8c3d92ff3f6 100644
> >>>> --- a/kernel/sched/fair.c
> >>>> +++ b/kernel/sched/fair.c
> >>>> @@ -7374,7 +7374,7 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu)
> >>>>         target = prev_cpu;
> >>>>
> >>>>         sync_entity_load_avg(&p->se);
> >>>> -       if (!uclamp_task_util(p, p_util_min, p_util_max))
> >>>> +       if (!uclamp_task_util(p, p_util_min, p_util_max) && p_util_max != 0)
> >>>
> >>> The below should do the same without testing twice p_util_max:
> >>> uclamp_task_util(p, p_util_min, ULONG_MAX)
> >>
> >> Since uclamp_task_util() is only used here and we don't want to test for
> >> capping to 0 anymore, why not just get rid of this function and use:
> >>
> >>   !(task_util_est(p) || p_util_min)
> >
> > That would be better, yes!
> >
> > Question for you and Vincent. Do we really want this optimization? I started
> > with removing it - then erred on the conservative side and kept it.
>
> Hard to say ... at least we know that util=0 will have absolutely no
> effect on task placement. So we can spare the heavy EAS algorithm in
> this case for sure.
>
> > I don't know how often we hit this case and I didn't see any benchmark run to
> > be able to verify anything when I looked at the history.
>
> There are very few EAS wakeups with `task_util_est(p) = 0`. Probably not
> relevant.
>
> > It seems helpful in theory - but why we save something if we ignore 0 but not
> > 1 which I suspect will not produce a significant delta either.
>
> True, it's hard to find the real line of significance here.
>
> > I don't mind keeping it - but I think worth thinking if it is really adding
> > much.
>
> I would keep it and just remove uclamp_task_util(). We still have a lot
> of uclamp/util related functions, we should try to keep the number as
> low as possible. Just checked it, this check has been there from the
> beginning of EAS.

Yes make sense to keep the test as proposed by Dietmar and save the cycles

>

Re: [PATCH v2 2/3] sched/uclamp: Ignore (util == 0) optimization in feec() when p_util_max = 0

Posted by Qais Yousef 2 years, 7 months ago

On 02/14/23 13:47, Dietmar Eggemann wrote:
> On 11/02/2023 19:01, Qais Yousef wrote:
> > On 02/08/23 12:52, Dietmar Eggemann wrote:
> >> On 07/02/2023 11:04, Vincent Guittot wrote:
> >>> On Sun, 5 Feb 2023 at 23:43, Qais Yousef <qyousef@layalina.io> wrote:
> >>>>
> >>>> find_energy_efficient_cpu() bails out early if effective util of the
> >>>> task is 0. When uclamp is being used, this could lead to wrong decisions
> >>>> when uclamp_max is set to 0. Cater for that.
> >>
> >> IMHO this needs a little bit more explanation. Someone could argue that
> >> 'util > 0, uclamp_min=0, uclamp_max=0' is a valid setup for a task which
> >> should let it appear as a task with 0 util (capped to 0).
> > 
> > You want me to explain the purpose of the optimization then?
> > 
> > The optimization skips energy calculation when util is 0 because the delta will
> > be 0. But when uclamp_max = 0 util is not really 0 - consequently  the delta
> 
> I would say:
> 
> s/really/necessarily
> s/delta/energy delta

+1

> 
> > will not be 0.
> > 
> > Would such an explanation clarify things better?
> 
> Yes. It key to understand that there is a 2-step process. First,
> admittance to a possible target (util and uclamp) and second, energy
> diff based target-selection (util).
> 
> >>>> Fixes: d81304bc6193 ("sched/uclamp: Cater for uclamp in find_energy_efficient_cpu()'s early exit condition")
> >>>> Signed-off-by: Qais Yousef <qyousef@layalina.io>
> >>>> ---
> >>>>  kernel/sched/fair.c | 2 +-
> >>>>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>>>
> >>>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> >>>> index 7a21ee74139f..a8c3d92ff3f6 100644
> >>>> --- a/kernel/sched/fair.c
> >>>> +++ b/kernel/sched/fair.c
> >>>> @@ -7374,7 +7374,7 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu)
> >>>>         target = prev_cpu;
> >>>>
> >>>>         sync_entity_load_avg(&p->se);
> >>>> -       if (!uclamp_task_util(p, p_util_min, p_util_max))
> >>>> +       if (!uclamp_task_util(p, p_util_min, p_util_max) && p_util_max != 0)
> >>>
> >>> The below should do the same without testing twice p_util_max:
> >>> uclamp_task_util(p, p_util_min, ULONG_MAX)
> >>
> >> Since uclamp_task_util() is only used here and we don't want to test for
> >> capping to 0 anymore, why not just get rid of this function and use:
> >>
> >>   !(task_util_est(p) || p_util_min)
> > 
> > That would be better, yes!
> > 
> > Question for you and Vincent. Do we really want this optimization? I started
> > with removing it - then erred on the conservative side and kept it.
> 
> Hard to say ... at least we know that util=0 will have absolutely no
> effect on task placement. So we can spare the heavy EAS algorithm in
> this case for sure.
> 
> > I don't know how often we hit this case and I didn't see any benchmark run to
> > be able to verify anything when I looked at the history.
> 
> There are very few EAS wakeups with `task_util_est(p) = 0`. Probably not
> relevant.
> 
> > It seems helpful in theory - but why we save something if we ignore 0 but not
> > 1 which I suspect will not produce a significant delta either.
> 
> True, it's hard to find the real line of significance here.
> 
> > I don't mind keeping it - but I think worth thinking if it is really adding
> > much.
> 
> I would keep it and just remove uclamp_task_util(). We still have a lot
> of uclamp/util related functions, we should try to keep the number as
> low as possible. Just checked it, this check has been there from the
> beginning of EAS.

Yeah I looked at the history and it was always there.

I'll update with the new check and update the commit message too.


Thanks!

--
Qais Yousef