1. Patchew
  2. linux
  3. [PATCH 0/3] sched: Miscellaneous isolation related cleanups
  4. View patch

[PATCH 2/3] sched/fair: Use HK_TYPE_SCHED housekeeping CPUs

Waiman Long posted 3 patches 1 year, 5 months ago
There is a newer version of this series
[PATCH 2/3] sched/fair: Use HK_TYPE_SCHED housekeeping CPUs
Posted by Waiman Long 1 year, 5 months ago 
As the previous commit has enabled the setting of HK_TYPE_SCHED
housekeeping CPUs in nohz_full setup, we can now use the more aptly
named HK_TYPE_SCHED housekeeping CPUs instead of HK_TYPE_MISC.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/sched/fair.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 6be618110885..0350667f5ce8 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -12128,16 +12128,13 @@ static inline int on_null_domain(struct rq *rq)
  * - When one of the busy CPUs notices that there may be an idle rebalancing
  *   needed, they will kick the idle load balancer, which then does idle
  *   load balancing for all the idle CPUs.
- *
- * - HK_TYPE_MISC CPUs are used for this task, because HK_TYPE_SCHED is not set
- *   anywhere yet.
  */
 static inline int find_new_ilb(void)
 {
 	const struct cpumask *hk_mask;
 	int ilb_cpu;
 
-	hk_mask = housekeeping_cpumask(HK_TYPE_MISC);
+	hk_mask = housekeeping_cpumask(HK_TYPE_SCHED);
 
 	for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) {
 
@@ -12155,7 +12152,7 @@ static inline int find_new_ilb(void)
  * Kick a CPU to do the NOHZ balancing, if it is time for it, via a cross-CPU
  * SMP function call (IPI).
  *
- * We pick the first idle CPU in the HK_TYPE_MISC housekeeping set (if there is one).
+ * We pick the first idle CPU in the HK_TYPE_SCHED housekeeping set (if there is one).
  */
 static void kick_ilb(unsigned int flags)
 {
-- 
2.43.5
Re: [PATCH 2/3] sched/fair: Use HK_TYPE_SCHED housekeeping CPUs
Posted by Frederic Weisbecker 1 year, 5 months ago 
Le Sun, Aug 18, 2024 at 07:45:19PM -0400, Waiman Long a écrit :
> As the previous commit has enabled the setting of HK_TYPE_SCHED
> housekeeping CPUs in nohz_full setup, we can now use the more aptly
> named HK_TYPE_SCHED housekeeping CPUs instead of HK_TYPE_MISC.
> 
> Signed-off-by: Waiman Long <longman@redhat.com>

Can we instead merge HK_FLAG_TICK, HK_FLAG_WQ, HK_FLAG_TIMER, HK_FLAG_RCU,
HK_FLAG_MISC and HK_FLAG_KTHREAD into a single
HK_FLAG_KERNEL_NOISE / HK_TYPE_KERNEL_NOISE ?

Thanks.

> ---
>  kernel/sched/fair.c | 7 ++-----
>  1 file changed, 2 insertions(+), 5 deletions(-)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 6be618110885..0350667f5ce8 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -12128,16 +12128,13 @@ static inline int on_null_domain(struct rq *rq)
>   * - When one of the busy CPUs notices that there may be an idle rebalancing
>   *   needed, they will kick the idle load balancer, which then does idle
>   *   load balancing for all the idle CPUs.
> - *
> - * - HK_TYPE_MISC CPUs are used for this task, because HK_TYPE_SCHED is not set
> - *   anywhere yet.
>   */
>  static inline int find_new_ilb(void)
>  {
>  	const struct cpumask *hk_mask;
>  	int ilb_cpu;
>  
> -	hk_mask = housekeeping_cpumask(HK_TYPE_MISC);
> +	hk_mask = housekeeping_cpumask(HK_TYPE_SCHED);
>  
>  	for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) {
>  
> @@ -12155,7 +12152,7 @@ static inline int find_new_ilb(void)
>   * Kick a CPU to do the NOHZ balancing, if it is time for it, via a cross-CPU
>   * SMP function call (IPI).
>   *
> - * We pick the first idle CPU in the HK_TYPE_MISC housekeeping set (if there is one).
> + * We pick the first idle CPU in the HK_TYPE_SCHED housekeeping set (if there is one).
>   */
>  static void kick_ilb(unsigned int flags)
>  {
> -- 
> 2.43.5
>
Re: [PATCH 2/3] sched/fair: Use HK_TYPE_SCHED housekeeping CPUs
Posted by Waiman Long 1 year, 5 months ago 
On 9/3/24 09:12, Frederic Weisbecker wrote:
> Le Sun, Aug 18, 2024 at 07:45:19PM -0400, Waiman Long a écrit :
>> As the previous commit has enabled the setting of HK_TYPE_SCHED
>> housekeeping CPUs in nohz_full setup, we can now use the more aptly
>> named HK_TYPE_SCHED housekeeping CPUs instead of HK_TYPE_MISC.
>>
>> Signed-off-by: Waiman Long <longman@redhat.com>
> Can we instead merge HK_FLAG_TICK, HK_FLAG_WQ, HK_FLAG_TIMER, HK_FLAG_RCU,
> HK_FLAG_MISC and HK_FLAG_KTHREAD into a single
> HK_FLAG_KERNEL_NOISE / HK_TYPE_KERNEL_NOISE ?

Sure. I am open to new name as I am not good on that. 
HK_FLAG_KERNEL_NOISE does make sense.

Cheers,
Longman

>
> Thanks.
>
>> ---
>>   kernel/sched/fair.c | 7 ++-----
>>   1 file changed, 2 insertions(+), 5 deletions(-)
>>
>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>> index 6be618110885..0350667f5ce8 100644
>> --- a/kernel/sched/fair.c
>> +++ b/kernel/sched/fair.c
>> @@ -12128,16 +12128,13 @@ static inline int on_null_domain(struct rq *rq)
>>    * - When one of the busy CPUs notices that there may be an idle rebalancing
>>    *   needed, they will kick the idle load balancer, which then does idle
>>    *   load balancing for all the idle CPUs.
>> - *
>> - * - HK_TYPE_MISC CPUs are used for this task, because HK_TYPE_SCHED is not set
>> - *   anywhere yet.
>>    */
>>   static inline int find_new_ilb(void)
>>   {
>>   	const struct cpumask *hk_mask;
>>   	int ilb_cpu;
>>   
>> -	hk_mask = housekeeping_cpumask(HK_TYPE_MISC);
>> +	hk_mask = housekeeping_cpumask(HK_TYPE_SCHED);
>>   
>>   	for_each_cpu_and(ilb_cpu, nohz.idle_cpus_mask, hk_mask) {
>>   
>> @@ -12155,7 +12152,7 @@ static inline int find_new_ilb(void)
>>    * Kick a CPU to do the NOHZ balancing, if it is time for it, via a cross-CPU
>>    * SMP function call (IPI).
>>    *
>> - * We pick the first idle CPU in the HK_TYPE_MISC housekeeping set (if there is one).
>> + * We pick the first idle CPU in the HK_TYPE_SCHED housekeeping set (if there is one).
>>    */
>>   static void kick_ilb(unsigned int flags)
>>   {
>> -- 
>> 2.43.5
>>
Re: [PATCH 2/3] sched/fair: Use HK_TYPE_SCHED housekeeping CPUs
Posted by Waiman Long 1 year, 5 months ago 
On 9/3/24 09:53, Waiman Long wrote:
>
> On 9/3/24 09:12, Frederic Weisbecker wrote:
>> Le Sun, Aug 18, 2024 at 07:45:19PM -0400, Waiman Long a écrit :
>>> As the previous commit has enabled the setting of HK_TYPE_SCHED
>>> housekeeping CPUs in nohz_full setup, we can now use the more aptly
>>> named HK_TYPE_SCHED housekeeping CPUs instead of HK_TYPE_MISC.
>>>
>>> Signed-off-by: Waiman Long <longman@redhat.com>
>> Can we instead merge HK_FLAG_TICK, HK_FLAG_WQ, HK_FLAG_TIMER, 
>> HK_FLAG_RCU,
>> HK_FLAG_MISC and HK_FLAG_KTHREAD into a single
>> HK_FLAG_KERNEL_NOISE / HK_TYPE_KERNEL_NOISE ?
>
> Sure. I am open to new name as I am not good on that. 
> HK_FLAG_KERNEL_NOISE does make sense.

I can't merge HK_FLAG_TICK just yet as it can be independently set by 
using isolcpus which can set just HK_TYPE_TICK, HK_TYPE_DOMAIN or 
HK_TYPE_MANAGED_IRQ. That is the reason why they are still separate.

Cheers,
Longman
Re: [PATCH 2/3] sched/fair: Use HK_TYPE_SCHED housekeeping CPUs
Posted by Frederic Weisbecker 1 year, 5 months ago 
Le Wed, Sep 04, 2024 at 10:54:15AM -0400, Waiman Long a écrit :
> 
> On 9/3/24 09:53, Waiman Long wrote:
> > 
> > On 9/3/24 09:12, Frederic Weisbecker wrote:
> > > Le Sun, Aug 18, 2024 at 07:45:19PM -0400, Waiman Long a écrit :
> > > > As the previous commit has enabled the setting of HK_TYPE_SCHED
> > > > housekeeping CPUs in nohz_full setup, we can now use the more aptly
> > > > named HK_TYPE_SCHED housekeeping CPUs instead of HK_TYPE_MISC.
> > > > 
> > > > Signed-off-by: Waiman Long <longman@redhat.com>
> > > Can we instead merge HK_FLAG_TICK, HK_FLAG_WQ, HK_FLAG_TIMER,
> > > HK_FLAG_RCU,
> > > HK_FLAG_MISC and HK_FLAG_KTHREAD into a single
> > > HK_FLAG_KERNEL_NOISE / HK_TYPE_KERNEL_NOISE ?
> > 
> > Sure. I am open to new name as I am not good on that.
> > HK_FLAG_KERNEL_NOISE does make sense.
> 
> I can't merge HK_FLAG_TICK just yet as it can be independently set by using
> isolcpus which can set just HK_TYPE_TICK, HK_TYPE_DOMAIN or
> HK_TYPE_MANAGED_IRQ. That is the reason why they are still separate.

Ah I think we really need to unify behaviours of isolcpus=nohz and nohz_full=

diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 5891e715f00d..9bc8fbf326cc 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -212,7 +212,8 @@ static int __init housekeeping_isolcpus_setup(char *str)
 	while (isalpha(*str)) {
 		if (!strncmp(str, "nohz,", 5)) {
 			str += 5;
-			flags |= HK_FLAG_TICK;
+			flags |= HK_FLAG_TICK | HK_FLAG_WQ | HK_FLAG_TIMER | HK_FLAG_RCU |
+				 HK_FLAG_MISC | HK_FLAG_KTHREAD;
 			continue;
 		}
Re: [PATCH 2/3] sched/fair: Use HK_TYPE_SCHED housekeeping CPUs
Posted by Waiman Long 1 year, 5 months ago 
On 9/6/24 08:53, Frederic Weisbecker wrote:
> Le Wed, Sep 04, 2024 at 10:54:15AM -0400, Waiman Long a écrit :
>> On 9/3/24 09:53, Waiman Long wrote:
>>> On 9/3/24 09:12, Frederic Weisbecker wrote:
>>>> Le Sun, Aug 18, 2024 at 07:45:19PM -0400, Waiman Long a écrit :
>>>>> As the previous commit has enabled the setting of HK_TYPE_SCHED
>>>>> housekeeping CPUs in nohz_full setup, we can now use the more aptly
>>>>> named HK_TYPE_SCHED housekeeping CPUs instead of HK_TYPE_MISC.
>>>>>
>>>>> Signed-off-by: Waiman Long <longman@redhat.com>
>>>> Can we instead merge HK_FLAG_TICK, HK_FLAG_WQ, HK_FLAG_TIMER,
>>>> HK_FLAG_RCU,
>>>> HK_FLAG_MISC and HK_FLAG_KTHREAD into a single
>>>> HK_FLAG_KERNEL_NOISE / HK_TYPE_KERNEL_NOISE ?
>>> Sure. I am open to new name as I am not good on that.
>>> HK_FLAG_KERNEL_NOISE does make sense.
>> I can't merge HK_FLAG_TICK just yet as it can be independently set by using
>> isolcpus which can set just HK_TYPE_TICK, HK_TYPE_DOMAIN or
>> HK_TYPE_MANAGED_IRQ. That is the reason why they are still separate.
> Ah I think we really need to unify behaviours of isolcpus=nohz and nohz_full=
>
> diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
> index 5891e715f00d..9bc8fbf326cc 100644
> --- a/kernel/sched/isolation.c
> +++ b/kernel/sched/isolation.c
> @@ -212,7 +212,8 @@ static int __init housekeeping_isolcpus_setup(char *str)
>   	while (isalpha(*str)) {
>   		if (!strncmp(str, "nohz,", 5)) {
>   			str += 5;
> -			flags |= HK_FLAG_TICK;
> +			flags |= HK_FLAG_TICK | HK_FLAG_WQ | HK_FLAG_TIMER | HK_FLAG_RCU |
> +				 HK_FLAG_MISC | HK_FLAG_KTHREAD;
>   			continue;
>   		}

I am not against it, but it is a change in userspace behavior. How about 
we introduce "isolcpus=nohz_full" option that essentially combines 
isolcpus and nohz_full in one option? That can be a separate patch.

Cheers,
Longman

Patchew 2.1-devel-b67e4c2

© 2016 - 2026 Red Hat, Inc.