This patchset restarts the work about adding a latency nice priority to
describe the latency tolerance of cfs tasks.

The patches [1-4] have been done by Parth:
https://lore.kernel.org/lkml/20200228090755.22829-1-parth@linux.ibm.com/

I have just rebased and moved the set of latency priority outside the
priority update. I have removed the reviewed tag because the patches
are 2 years old.

The patches [5-7] use latency nice priority to decide if a cfs task can
preempt the current running task. Patch 5 gives some tests results with
cyclictests and hackbench to highlight the benefit of latency nice
priority for short interactive task or long intensive tasks.


Change since v1:
- fix typo
- move some codes in the right patch to make bisect happy
- simplify and fixed how the weight is computed
- added support of sched core patch 7

Parth Shah (4):
  sched: Introduce latency-nice as a per-task attribute
  sched/core: Propagate parent task's latency requirements to the child
    task
  sched: Allow sched_{get,set}attr to change latency_nice of the task
  sched/core: Add permission checks for setting the latency_nice value

Vincent Guittot (3):
  sched/fair: Take into account latency nice at wakeup
  sched/fair: Add sched group latency support
  sched/core: support latency nice with sched core

 include/linux/sched.h            |   3 +
 include/uapi/linux/sched.h       |   4 +-
 include/uapi/linux/sched/types.h |  19 ++++++
 init/init_task.c                 |   1 +
 kernel/sched/core.c              |  90 ++++++++++++++++++++++++++
 kernel/sched/debug.c             |   1 +
 kernel/sched/fair.c              | 105 ++++++++++++++++++++++++++++++-
 kernel/sched/sched.h             |  34 ++++++++++
 tools/include/uapi/linux/sched.h |   4 +-
 9 files changed, 257 insertions(+), 4 deletions(-)

-- 
2.17.1

On Thu, 2022-05-12 at 18:35 +0200, Vincent Guittot wrote:
> This patchset restarts the work about adding a latency nice priority to
> describe the latency tolerance of cfs tasks.
> 
> The patches [1-4] have been done by Parth:
> https://lore.kernel.org/lkml/20200228090755.22829-1-parth@linux.ibm.com/
> 
> I have just rebased and moved the set of latency priority outside the
> priority update. I have removed the reviewed tag because the patches
> are 2 years old.
> 

Vincent,

Thanks for introducing the feature again, which is much needed.  I am trying
to look at the problem again from usage point of view. And wonder if
there are ways to make the latency_nice knob easier to use.

The latency nice value here is relative.  A latency sensitive task
may not tell if setting the latency_nice to -5, or to -10 is good enough.
It depends on what other tasks are setting their latency_nice value to.
What a task does know is what it it doing and its characteristics.  
For instance for client tasks, we may have categories such as

Task Category					latency_nice_range
-------------                                   ------------------
urgent						-19 to -16
media playback					-15 to -11
interactive (e.g.pressing key)  		-10 to -6
normal						-5  to  9
background					 10  to 15
opportunistic soaker task (sched_idle class)	 16 to  20

And we could allow a task to set attribute of which task category applies
to it and the OS can set a default latency nice value in its task category.   
So a task can just declare itself what kind of task it is, and not worry about
actually setting a latency nice value which it may not know
what is appopriate. 
If needed, a task could still adjust its latency nice value within the range to
differentiate itself in a task category. And we will prevent
a task from seeting inappropriate latency nice value out of the right range.

Knowing a task characteristics will also be helpful with other
scheduling decisions, like placing a task on a more high performing
core in hetero systems.  

I think the missing piece here is a way for a task to declare 
what kind of task it is.  I think that will make things easier.

Tim

> The patches [5-7] use latency nice priority to decide if a cfs task can
> preempt the current running task. Patch 5 gives some tests results with
> cyclictests and hackbench to highlight the benefit of latency nice
> priority for short interactive task or long intensive tasks.
> 
> 
> Change since v1:
> - fix typo
> - move some codes in the right patch to make bisect happy
> - simplify and fixed how the weight is computed
> - added support of sched core patch 7
> 
> Parth Shah (4):
>   sched: Introduce latency-nice as a per-task attribute
>   sched/core: Propagate parent task's latency requirements to the child
>     task
>   sched: Allow sched_{get,set}attr to change latency_nice of the task
>   sched/core: Add permission checks for setting the latency_nice value
> 
> Vincent Guittot (3):
>   sched/fair: Take into account latency nice at wakeup
>   sched/fair: Add sched group latency support
>   sched/core: support latency nice with sched core
> 
>  include/linux/sched.h            |   3 +
>  include/uapi/linux/sched.h       |   4 +-
>  include/uapi/linux/sched/types.h |  19 ++++++
>  init/init_task.c                 |   1 +
>  kernel/sched/core.c              |  90 ++++++++++++++++++++++++++
>  kernel/sched/debug.c             |   1 +
>  kernel/sched/fair.c              | 105 ++++++++++++++++++++++++++++++-
>  kernel/sched/sched.h             |  34 ++++++++++
>  tools/include/uapi/linux/sched.h |   4 +-
>  9 files changed, 257 insertions(+), 4 deletions(-)
>

On Fri, 13 May 2022 at 23:44, Tim Chen <tim.c.chen@linux.intel.com> wrote:
>
> On Thu, 2022-05-12 at 18:35 +0200, Vincent Guittot wrote:
> > This patchset restarts the work about adding a latency nice priority to
> > describe the latency tolerance of cfs tasks.
> >
> > The patches [1-4] have been done by Parth:
> > https://lore.kernel.org/lkml/20200228090755.22829-1-parth@linux.ibm.com/
> >
> > I have just rebased and moved the set of latency priority outside the
> > priority update. I have removed the reviewed tag because the patches
> > are 2 years old.
> >
>
> Vincent,
>
> Thanks for introducing the feature again, which is much needed.  I am trying
> to look at the problem again from usage point of view. And wonder if
> there are ways to make the latency_nice knob easier to use.
>
> The latency nice value here is relative.  A latency sensitive task
> may not tell if setting the latency_nice to -5, or to -10 is good enough.
> It depends on what other tasks are setting their latency_nice value to.
> What a task does know is what it it doing and its characteristics.
> For instance for client tasks, we may have categories such as
>
> Task Category                                   latency_nice_range
> -------------                                   ------------------
> urgent                                          -19 to -16
> media playback                                  -15 to -11
> interactive (e.g.pressing key)                  -10 to -6
> normal                                          -5  to  9
> background                                       10  to 15
> opportunistic soaker task (sched_idle class)     16 to  20
>
> And we could allow a task to set attribute of which task category applies
> to it and the OS can set a default latency nice value in its task category.
> So a task can just declare itself what kind of task it is, and not worry about
> actually setting a latency nice value which it may not know
> what is appopriate.
> If needed, a task could still adjust its latency nice value within the range to
> differentiate itself in a task category. And we will prevent
> a task from seeting inappropriate latency nice value out of the right range.

The description above make sense but I'm not sure this should be put
as part of the interface but more in the documentation to describe how
system can make use of nice_latency
>
> Knowing a task characteristics will also be helpful with other
> scheduling decisions, like placing a task on a more high performing
> core in hetero systems.

Ok so you would like a more general interface than an latency
interface but a way to set some attributes to a task so we can make
smarter decision

>
> I think the missing piece here is a way for a task to declare
> what kind of task it is.  I think that will make things easier.
>
> Tim
>
> > The patches [5-7] use latency nice priority to decide if a cfs task can
> > preempt the current running task. Patch 5 gives some tests results with
> > cyclictests and hackbench to highlight the benefit of latency nice
> > priority for short interactive task or long intensive tasks.
> >
> >
> > Change since v1:
> > - fix typo
> > - move some codes in the right patch to make bisect happy
> > - simplify and fixed how the weight is computed
> > - added support of sched core patch 7
> >
> > Parth Shah (4):
> >   sched: Introduce latency-nice as a per-task attribute
> >   sched/core: Propagate parent task's latency requirements to the child
> >     task
> >   sched: Allow sched_{get,set}attr to change latency_nice of the task
> >   sched/core: Add permission checks for setting the latency_nice value
> >
> > Vincent Guittot (3):
> >   sched/fair: Take into account latency nice at wakeup
> >   sched/fair: Add sched group latency support
> >   sched/core: support latency nice with sched core
> >
> >  include/linux/sched.h            |   3 +
> >  include/uapi/linux/sched.h       |   4 +-
> >  include/uapi/linux/sched/types.h |  19 ++++++
> >  init/init_task.c                 |   1 +
> >  kernel/sched/core.c              |  90 ++++++++++++++++++++++++++
> >  kernel/sched/debug.c             |   1 +
> >  kernel/sched/fair.c              | 105 ++++++++++++++++++++++++++++++-
> >  kernel/sched/sched.h             |  34 ++++++++++
> >  tools/include/uapi/linux/sched.h |   4 +-
> >  9 files changed, 257 insertions(+), 4 deletions(-)
> >
>

On 5/19/22 10:16 AM, Vincent Guittot wrote:
> On Fri, 13 May 2022 at 23:44, Tim Chen <tim.c.chen@linux.intel.com> wrote:
>>
>> On Thu, 2022-05-12 at 18:35 +0200, Vincent Guittot wrote:
>>> This patchset restarts the work about adding a latency nice priority to
>>> describe the latency tolerance of cfs tasks.
>>>
>>> The patches [1-4] have been done by Parth:
>>> https://lore.kernel.org/lkml/20200228090755.22829-1-parth@linux.ibm.com/
>>>
>>> I have just rebased and moved the set of latency priority outside the
>>> priority update. I have removed the reviewed tag because the patches
>>> are 2 years old.
>>>
>>
>> Vincent,
>>
>> Thanks for introducing the feature again, which is much needed.  I am trying
>> to look at the problem again from usage point of view. And wonder if
>> there are ways to make the latency_nice knob easier to use.
>>
>> The latency nice value here is relative.  A latency sensitive task
>> may not tell if setting the latency_nice to -5, or to -10 is good enough.
>> It depends on what other tasks are setting their latency_nice value to.
>> What a task does know is what it it doing and its characteristics.
>> For instance for client tasks, we may have categories such as
>>
>> Task Category                                   latency_nice_range
>> -------------                                   ------------------
>> urgent                                          -19 to -16
>> media playback                                  -15 to -11
>> interactive (e.g.pressing key)                  -10 to -6
>> normal                                          -5  to  9
>> background                                       10  to 15
>> opportunistic soaker task (sched_idle class)     16 to  20
>>
>> And we could allow a task to set attribute of which task category applies
>> to it and the OS can set a default latency nice value in its task category.
>> So a task can just declare itself what kind of task it is, and not worry about
>> actually setting a latency nice value which it may not know
>> what is appopriate.
>> If needed, a task could still adjust its latency nice value within the range to
>> differentiate itself in a task category. And we will prevent
>> a task from seeting inappropriate latency nice value out of the right range.
> 
> The description above make sense but I'm not sure this should be put
> as part of the interface but more in the documentation to describe how
> system can make use of nice_latency
>>
>> Knowing a task characteristics will also be helpful with other
>> scheduling decisions, like placing a task on a more high performing
>> core in hetero systems.
> 
> Ok so you would like a more general interface than an latency
> interface but a way to set some attributes to a task so we can make
> smarter decision

The original definition of latency nice was as a task attribute describing the latency sensitivity of the task. The fact 
that it was mapped to 'nice' values created too much granularity and made it look more like a tuning knob than a 
statement about the characteristics of the task as intended.

> 
>>
>> I think the missing piece here is a way for a task to declare
>> what kind of task it is.  I think that will make things easier.

A classification of tasks into categories would be useful, but perhaps one level up in a user space tool or a user's 
head (ie docs). For any of the categories you describe, there may be a number of per-task attributes beyond latency 
sensitivity needed to capture the task characteristics you mention and ideally would be set in specific ways. Say 'nice' 
values, oom kill, etc. And others may make sense in the future, like say NUMA sensitivity, etc.

Basically, a category can map to a bunch of desired default values for various scheduler visible task attributes.

Now you could also take the idea in the other direction where you set a "category value" for a task and have the kernel 
pick the other attribute defaults like 'nice' that would typically apply to tasks in the category, but I think letting 
user space figure stuff out and then set low level kernel task attributes primitives is cleaner.

-chrish

On Thu, 2022-05-19 at 14:14 -0400, Chris Hyser wrote:
> On 5/19/22 10:16 AM, Vincent Guittot wrote:
> > On Fri, 13 May 2022 at 23:44, Tim Chen <tim.c.chen@linux.intel.com> wrote:
> > > On Thu, 2022-05-12 at 18:35 +0200, Vincent Guittot wrote:
> > > > This patchset restarts the work about adding a latency nice priority to
> > > > describe the latency tolerance of cfs tasks.
> > > > 
> > > > The patches [1-4] have been done by Parth:
> > > > https://lore.kernel.org/lkml/20200228090755.22829-1-parth@linux.ibm.com/
> > > > 
> > > > I have just rebased and moved the set of latency priority outside the
> > > > priority update. I have removed the reviewed tag because the patches
> > > > are 2 years old.
> > > > 
> > > 
> > > Vincent,
> > > 
> > > Thanks for introducing the feature again, which is much needed.  I am trying
> > > to look at the problem again from usage point of view. And wonder if
> > > there are ways to make the latency_nice knob easier to use.
> > > 
> > > The latency nice value here is relative.  A latency sensitive task
> > > may not tell if setting the latency_nice to -5, or to -10 is good enough.
> > > It depends on what other tasks are setting their latency_nice value to.
> > > What a task does know is what it it doing and its characteristics.
> > > For instance for client tasks, we may have categories such as
> > > 
> > > Task Category                                   latency_nice_range
> > > -------------                                   ------------------
> > > urgent                                          -19 to -16
> > > media playback                                  -15 to -11
> > > interactive (e.g.pressing key)                  -10 to -6
> > > normal                                          -5  to  9
> > > background                                       10  to 15
> > > opportunistic soaker task (sched_idle class)     16 to  20
> > > 
> > > And we could allow a task to set attribute of which task category applies
> > > to it and the OS can set a default latency nice value in its task category.
> > > So a task can just declare itself what kind of task it is, and not worry about
> > > actually setting a latency nice value which it may not know
> > > what is appopriate.
> > > If needed, a task could still adjust its latency nice value within the range to
> > > differentiate itself in a task category. And we will prevent
> > > a task from seeting inappropriate latency nice value out of the right range.
> > 
> > The description above make sense but I'm not sure this should be put
> > as part of the interface but more in the documentation to describe how
> > system can make use of nice_latency
> > > Knowing a task characteristics will also be helpful with other
> > > scheduling decisions, like placing a task on a more high performing
> > > core in hetero systems.
> > 
> > Ok so you would like a more general interface than an latency
> > interface but a way to set some attributes to a task so we can make
> > smarter decision
> 
> The original definition of latency nice was as a task attribute describing the latency sensitivity of the task. The fact 
> that it was mapped to 'nice' values created too much granularity and made it look more like a tuning knob than a 
> statement about the characteristics of the task as intended.
> 
> > > I think the missing piece here is a way for a task to declare
> > > what kind of task it is.  I think that will make things easier.
> 
> A classification of tasks into categories would be useful, but perhaps one level up in a user space tool or a user's 
> head (ie docs). For any of the categories you describe, there may be a number of per-task attributes beyond latency 
> sensitivity needed to capture the task characteristics you mention and ideally would be set in specific ways. Say 'nice' 
> values, oom kill, etc. And others may make sense in the future, like say NUMA sensitivity, etc.
> 
> Basically, a category can map to a bunch of desired default values for various scheduler visible task attributes.

Yes.  I think having a default value for each category will make the setting of the attributes
consistent and life of a task simpler.

Without guidance, one media playback task can set its latency_nice to be -5, 
and another interactive task doing computation and displaying results to user
set its latency_nice to be -6. We would have the interactive task running ahead of the media playback,
which is undesired.  Just letting each task set its own latency_nice value will not achieve the
desired effect.  We need guidance on what attribute value a certain task category 
should use (as in documentation that Vincent mentioned).

Or let OS set the attribute to some sensible value if it knows the task category.

> 
> Now you could also take the idea in the other direction where you set a "category value" for a task and have the kernel 
> pick the other attribute defaults like 'nice' that would typically apply to tasks in the category, but I think letting 
> user space figure stuff out and then set low level kernel task attributes primitives is cleaner.

It is cleaner for the OS to give the knob to the user and say, "set it".
But users need guidance on what knob value they should use,
or even better, they don't need to worry about setting the knob.
I think it is easy for the user to set task category correctly, but
harder to set the low level knobs.

I can see that it is easy to set the latency_nice knob incorrectly.
The absolute value of the knob doesn't matter. Only the relative values
between tasks do. So we need all the tasks to agree on what latency-nice
to use for a task category.  

Tim


> 
> -chrish

On Fri, 20 May 2022 at 20:46, Tim Chen <tim.c.chen@linux.intel.com> wrote:
>
> On Thu, 2022-05-19 at 14:14 -0400, Chris Hyser wrote:
> > On 5/19/22 10:16 AM, Vincent Guittot wrote:
> > > On Fri, 13 May 2022 at 23:44, Tim Chen <tim.c.chen@linux.intel.com> wrote:
> > > > On Thu, 2022-05-12 at 18:35 +0200, Vincent Guittot wrote:
> > > > > This patchset restarts the work about adding a latency nice priority to
> > > > > describe the latency tolerance of cfs tasks.
> > > > >
> > > > > The patches [1-4] have been done by Parth:
> > > > > https://lore.kernel.org/lkml/20200228090755.22829-1-parth@linux.ibm.com/
> > > > >
> > > > > I have just rebased and moved the set of latency priority outside the
> > > > > priority update. I have removed the reviewed tag because the patches
> > > > > are 2 years old.
> > > > >
> > > >
> > > > Vincent,
> > > >
> > > > Thanks for introducing the feature again, which is much needed.  I am trying
> > > > to look at the problem again from usage point of view. And wonder if
> > > > there are ways to make the latency_nice knob easier to use.
> > > >
> > > > The latency nice value here is relative.  A latency sensitive task
> > > > may not tell if setting the latency_nice to -5, or to -10 is good enough.
> > > > It depends on what other tasks are setting their latency_nice value to.
> > > > What a task does know is what it it doing and its characteristics.
> > > > For instance for client tasks, we may have categories such as
> > > >
> > > > Task Category                                   latency_nice_range
> > > > -------------                                   ------------------
> > > > urgent                                          -19 to -16
> > > > media playback                                  -15 to -11
> > > > interactive (e.g.pressing key)                  -10 to -6
> > > > normal                                          -5  to  9
> > > > background                                       10  to 15
> > > > opportunistic soaker task (sched_idle class)     16 to  20
> > > >
> > > > And we could allow a task to set attribute of which task category applies
> > > > to it and the OS can set a default latency nice value in its task category.
> > > > So a task can just declare itself what kind of task it is, and not worry about
> > > > actually setting a latency nice value which it may not know
> > > > what is appopriate.
> > > > If needed, a task could still adjust its latency nice value within the range to
> > > > differentiate itself in a task category. And we will prevent
> > > > a task from seeting inappropriate latency nice value out of the right range.
> > >
> > > The description above make sense but I'm not sure this should be put
> > > as part of the interface but more in the documentation to describe how
> > > system can make use of nice_latency
> > > > Knowing a task characteristics will also be helpful with other
> > > > scheduling decisions, like placing a task on a more high performing
> > > > core in hetero systems.
> > >
> > > Ok so you would like a more general interface than an latency
> > > interface but a way to set some attributes to a task so we can make
> > > smarter decision
> >
> > The original definition of latency nice was as a task attribute describing the latency sensitivity of the task. The fact
> > that it was mapped to 'nice' values created too much granularity and made it look more like a tuning knob than a
> > statement about the characteristics of the task as intended.
> >
> > > > I think the missing piece here is a way for a task to declare
> > > > what kind of task it is.  I think that will make things easier.
> >
> > A classification of tasks into categories would be useful, but perhaps one level up in a user space tool or a user's
> > head (ie docs). For any of the categories you describe, there may be a number of per-task attributes beyond latency
> > sensitivity needed to capture the task characteristics you mention and ideally would be set in specific ways. Say 'nice'
> > values, oom kill, etc. And others may make sense in the future, like say NUMA sensitivity, etc.
> >
> > Basically, a category can map to a bunch of desired default values for various scheduler visible task attributes.
>
> Yes.  I think having a default value for each category will make the setting of the attributes
> consistent and life of a task simpler.
>
> Without guidance, one media playback task can set its latency_nice to be -5,
> and another interactive task doing computation and displaying results to user
> set its latency_nice to be -6. We would have the interactive task running ahead of the media playback,
> which is undesired.  Just letting each task set its own latency_nice value will not achieve the
> desired effect.  We need guidance on what attribute value a certain task category
> should use (as in documentation that Vincent mentioned).
>
> Or let OS set the attribute to some sensible value if it knows the task category.

I'm in favor of documentation that provides guidance rather than the
OS setting attribute because the values are relative to each others so
we will always find some exceptions to a default configuration for let
say media vs user interaction. With a documentation, someone can
always not follow it if he thinks it doesn't apply to his system
because of whatever the reason. With the OS setting attributes, this
exception will have to abuse its category to get the right priority vs
others

>
> >
> > Now you could also take the idea in the other direction where you set a "category value" for a task and have the kernel
> > pick the other attribute defaults like 'nice' that would typically apply to tasks in the category, but I think letting
> > user space figure stuff out and then set low level kernel task attributes primitives is cleaner.
>
> It is cleaner for the OS to give the knob to the user and say, "set it".
> But users need guidance on what knob value they should use,
> or even better, they don't need to worry about setting the knob.
> I think it is easy for the user to set task category correctly, but
> harder to set the low level knobs.
>
> I can see that it is easy to set the latency_nice knob incorrectly.
> The absolute value of the knob doesn't matter. Only the relative values
> between tasks do. So we need all the tasks to agree on what latency-nice
> to use for a task category.
>
> Tim
>
>
> >
> > -chrish
>

> > > > > media playback                                  -15 to -11

Isn't that what the RT scheduler is for....

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

From: Vincent Guittot
> Sent: 12 May 2022 17:35
> 
> This patchset restarts the work about adding a latency nice priority to
> describe the latency tolerance of cfs tasks.
> 
> The patches [1-4] have been done by Parth:
> https://lore.kernel.org/lkml/20200228090755.22829-1-parth@linux.ibm.com/
> 
> I have just rebased and moved the set of latency priority outside the
> priority update. I have removed the reviewed tag because the patches
> are 2 years old.
> 
> The patches [5-7] use latency nice priority to decide if a cfs task can
> preempt the current running task. Patch 5 gives some tests results with
> cyclictests and hackbench to highlight the benefit of latency nice
> priority for short interactive task or long intensive tasks.

I'd have thought the best way to reduce latency would be to look
harder for an idle cpu before trying to preempt the current task.

By far the worst case latency for a cfs task is waking it from an
rt task.
AFAICT the cpu selection algorithm is something like:
1) if the cpu it last ran on is idle, schedule it there.
2) if one of a small subset of cpu is idle run it there.
3) schedule on the current cpu after the rt thread exits.

Then there is the additional behaviour:
An rt thread (almost) always starts on the cpu it ran on last,
any running cfs thread is put on the q for that cpu.

This makes it very difficult to start a background cfs thread
from an active rt thread.
Quite often it won't migrate to an idle cpu until the timer
tick scheduler rebalance happens.

I think the search for an idle cpu is also limited when woken
by a cfs thread.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

On Fri, 13 May 2022 at 09:13, David Laight <David.Laight@aculab.com> wrote:
>
> From: Vincent Guittot
> > Sent: 12 May 2022 17:35
> >
> > This patchset restarts the work about adding a latency nice priority to
> > describe the latency tolerance of cfs tasks.
> >
> > The patches [1-4] have been done by Parth:
> > https://lore.kernel.org/lkml/20200228090755.22829-1-parth@linux.ibm.com/
> >
> > I have just rebased and moved the set of latency priority outside the
> > priority update. I have removed the reviewed tag because the patches
> > are 2 years old.
> >
> > The patches [5-7] use latency nice priority to decide if a cfs task can
> > preempt the current running task. Patch 5 gives some tests results with
> > cyclictests and hackbench to highlight the benefit of latency nice
> > priority for short interactive task or long intensive tasks.
>
> I'd have thought the best way to reduce latency would be to look
> harder for an idle cpu before trying to preempt the current task.

Although it's a good policy, this is not always true:
- The wakeup latency of an idle CPU can be significant for deep idle
state (several ms)
- The cost of looking for an idle CPU can be higher than preempting
current thread

>
> By far the worst case latency for a cfs task is waking it from an
> rt task.
> AFAICT the cpu selection algorithm is something like:
> 1) if the cpu it last ran on is idle, schedule it there.
> 2) if one of a small subset of cpu is idle run it there.

yes the LLC

> 3) schedule on the current cpu after the rt thread exits.

when prev and current cpu are differents, we start to select which one
will be the starting point for looking for an idle cpu

>
> Then there is the additional behaviour:
> An rt thread (almost) always starts on the cpu it ran on last,
> any running cfs thread is put on the q for that cpu.

and might even have to wait for other cfs threads to run 1st which is
where this patch want to improve in priority

>
> This makes it very difficult to start a background cfs thread
> from an active rt thread.
> Quite often it won't migrate to an idle cpu until the timer
> tick scheduler rebalance happens.

so you are speaking about idle cpu out of the LLC
But then, this cfs task should try to wakeup 1st on the new cpu and
not on the RT one

>
> I think the search for an idle cpu is also limited when woken
> by a cfs thread.
>
>         David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)
>